Uh oh!
There was an error while loading. Please reload this page.
[camera] Add support for zero-shutter-lag capture - #12374
Conversation
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
1041d87 to
5bc45d6Compare5bc45d6 to
a0bc225Comparea0bc225 to
ca4ff9cCompareAdds an opt-in zero-shutter-lag capture API for platforms that support it. Part of flutter/flutter#190598. ## What - `isZeroShutterLagSupported(int cameraId)` — support query, defaults to `false` - `setZeroShutterLagEnabled(int cameraId, bool enabled)` — defaults to a no-op ## Why non-breaking Defaults preserve today's behavior on every platform that does not override them, following the existing `setJpegImageQuality` (no-op default) and `getSupportedVideoStabilizationModes` (support query) patterns in this file.
Android implementation of the zero-shutter-lag API added in camera_platform_interface 2.14.0 (this bumps the constraint to ^2.14.0). Part of flutter/flutter#190598. Measured on a Galaxy S21 (SM-G991N): the captured frame lands 629 ms closer to the shutter tap, and shutter-to-file time drops 912 ms -> 639 ms (methodology and raw numbers in the issue). - `isZeroShutterLagSupported` -> [`CameraInfo.isZslSupported()`](https://developer.android.com/reference/androidx/camera/core/CameraInfo#isZslSupported()) - `setZeroShutterLagEnabled` -> [`ImageCapture.Builder.setCaptureMode`](https://developer.android.com/reference/androidx/camera/core/ImageCapture.Builder#setCaptureMode(int)) with [`CAPTURE_MODE_ZERO_SHUTTER_LAG`](https://developer.android.com/reference/androidx/camera/core/ImageCapture#CAPTURE_MODE_ZERO_SHUTTER_LAG) CameraX degrades on its own, and only for still capture. Per the CAPTURE_MODE_ZERO_SHUTTER_LAG reference above, the mode is disabled automatically when VideoCapture is bound, flash mode is not OFF, or an OEM extension is active. The granularity differs: flash falls back per shot ([`CaptureConfigAdapter.kt`](https://github.com/androidx/androidx/blob/androidx-main/camera/camera-camera2/src/main/java/androidx/camera/camera2/adapter/CaptureConfigAdapter.kt#L133-L138)), while a device without private reprocessing never activates the ZSL session at all ([`ZslControl.kt`](https://github.com/androidx/androidx/blob/androidx-main/camera/camera-camera2/src/main/java/androidx/camera/camera2/adapter/ZslControl.kt#L138-L150)). Apps need no special handling. <details><summary>Why the ImageCapture use case is recreated</summary> CameraX only accepts the capture mode at `Builder` time, so `setZeroShutterLagEnabled` unbinds and recreates the `ImageCapture` use case — the same approach the existing `setJpegImageQuality` uses for the same constraint. Both are refactored onto one shared `_recreateImageCapture()` helper, so both settings (`jpegQuality`, ZSL) are always reapplied together; `createCamera` applies them on initial construction as well. Regression tests cover both directions. </details> <details><summary>Experimental-API opt-in</summary> Both CameraX members are annotated [`@ExperimentalZeroShutterLag`](https://developer.android.com/reference/androidx/camera/core/ExperimentalZeroShutterLag), a `@RequiresOptIn` marker consumed per declaration. Handled with `@OptIn`, matching this plugin's existing opt-ins (`ExperimentalCamera2Interop`, `ExperimentalLensFacing`, `ExperimentalPersistentRecording`); no build configuration change and nothing reaches the Dart surface. Verified clean against the repo's lint setup (`warningsAsErrors` + baseline) with `gradlew lintDebug`. </details>
iOS implementation of the zero-shutter-lag API added in camera_platform_interface 2.14.0 (this bumps the constraint to ^2.14.0). Part of flutter/flutter#190598. ## What - `isZeroShutterLagSupported` -> [`AVCapturePhotoOutput.isZeroShutterLagSupported`](https://developer.apple.com/documentation/avfoundation/avcapturephotooutput/iszeroshutterlagsupported) (iOS 17+/macOS 14+; `false` on older versions) - `setZeroShutterLagEnabled` -> [`AVCapturePhotoOutput.isZeroShutterLagEnabled`](https://developer.apple.com/documentation/avfoundation/avcapturephotooutput/iszeroshutterlagenabled) ## Why explicit control when iOS auto-enables it Per the `AVCapturePhotoOutput.h` header, "For apps linked on or after iOS 17 zero shutter lag is automatically enabled when supported." The documented role of the setter is the explicit opt-out (WWDC23 session 10105: "you can set AVCapturePhotoOutput.isZeroShutterLagEnabled to false to opt out" - https://developer.apple.com/videos/play/wwdc2023/10105/), plus deterministic re-enable: support can be revoked by configuration changes, and when it is, the enabled state silently reverts. Because of that auto-enable, iOS 17+ starts with zero shutter lag on while Android starts with it off, so the same Dart call is an opt-out here and an opt-in there. No latency numbers are quoted for iOS: the platform already enables the mode by default, so this change is about making that state explicit and controllable, not about improving it. <details><summary>Guard and configuration timing</summary> The header states the property "may only be set to YES if zeroShutterLagSupported is YES, otherwise an NSInvalidArgumentException is thrown", and that changing it "requires a lengthy reconfiguration of the capture render pipeline", to be done within `beginConfiguration`/`commitConfiguration` while running. The implementation guards enabling on the support query and wraps the write in a configuration block. The requested state is kept (plugin- and camera-level) and reapplied after session reconfigurations and camera switches, which silently reset it — mirroring the settings-retention approach on the Android side. Availability is handled inside the `CapturePhotoOutput` protocol passthrough (`flutter`-prefixed members, matching the existing `CaptureDevice.flutterActiveFormat` pattern), so callers need no availability checks and pre-iOS 17 devices report unsupported / ignore writes. </details> Regenerating with pigeon 26.1.5 also reverts two pre-existing hand-edits to the generated files from flutter#11616 (a redundant Int32 fallback cast and a missing Dart doc comment).
Generated with the repo tool per the federated-plugin process (https://github.com/flutter/flutter/blob/master/docs/ecosystem/contributing/README.md#changing-federated-plugins): dart run script/tool/bin/flutter_plugin_tools.dart \ make-deps-path-based --target-dependencies=camera_platform_interface CI is expected to be red on the publish check only; that check clears when this commit is reverted after the platform interface change is published. Part of flutter/flutter#190598.
ca4ff9c to
e9735d7CompareThere was a problem hiding this comment.
Code Review
This pull request adds support for zero-shutter-lag still image capture to the Flutter camera plugin. It introduces isZeroShutterLagSupported and setZeroShutterLagEnabled to the camera_platform_interface and implements them in the Android (CameraX) and iOS (AVFoundation) implementations, along with corresponding tests and Pigeon updates. Feedback on the changes suggests awaiting the asynchronous isZslSupported call in the Android implementation to avoid potential static analysis errors.
Uh oh!
There was an error while loading. Please reload this page.
Combines the interface and both implementations in one PR per the federated plugin process; splits into an interface-first PR once the direction is approved. The API shape is a proposal for flutter/flutter#190598 — happy to reshape it however you prefer.
Adds explicit zero-shutter-lag control: an opt-in on Android, where the default capture mode leaves it off, and an opt-out on iOS, where the OS enables it automatically when the session supports it.
What's added
camera_platform_interface— two methods onCameraPlatform, modeled on the existingsetJpegImageQuality/getSupportedVideoStabilizationModespatterns. Defaults arefalse/ no-op, so nothing changes for platforms or apps that don't opt in:camera_android_camerax—CameraInfo.isZslSupported()andImageCapture.Builder.setCaptureMode(CAPTURE_MODE_ZERO_SHUTTER_LAG). The capture mode is construction-only, so the use case is recreated — the approachsetJpegImageQualityalready uses for the same constraint, now sharing one helper so the two settings can't drop each other.camera_avfoundation—isZeroShutterLagSupported/isZeroShutterLagEnabled(iOS 17+). Enabling is guarded on the support query (setting it while unsupported throwsNSInvalidArgumentException) and wrapped inbeginConfiguration/commitConfiguration. The requested state is reapplied after camera switches, which silently reset it. Apps that never call the setter are left untouched, so iOS keeps its automatic behavior on upgrade.Notes for reviewers
CAPTURE_MODE_ZERO_SHUTTER_LAGis@ExperimentalZeroShutterLag, handled with per-declaration@OptInlike this plugin's existingExperimentalCamera2Interop/ExperimentalLensFacing/ExperimentalPersistentRecording.gradlew lintis clean.Int32cast and a missing doc comment).make-deps-path-basedoverrides and is reverted before landing.On-device measurements
Android — Galaxy S21 (SM-G991N), 5 captures per mode, shutter fired via
adb shell input tap. A running millisecond stopwatch was photographed; the time visible in each saved photo shows how far behind the shutter tap the captured frame is.MINIMIZE_LATENCY(current)ZERO_SHUTTER_LAGiOS — iPhone (iOS 26.5),
ResolutionPreset.max, 5 captures per mode after discarding one warm-up capture. ComparesAVCapturePhoto.timestampagainst the host-clock time of thecapturePhotocall, so a negative value means the saved frame predates the call.With ZSL on, every captured frame predated the capture call. Support is session-dependent rather than device-wide: the same device reports supported at
ResolutionPreset.maxand unsupported atmedium.Since iOS enables this automatically, these numbers describe what the toggle controls — not an improvement this PR delivers on iOS.
Part of flutter/flutter#190598
Pre-Review Checklist
[shared_preferences]///).Footnotes
Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩↩2