Skip to content

feat(render): render an arbitrary frame range, with the segment's own audio - #169

Merged
LeadcodeDev merged 1 commit into
mainfrom
feat/frame-range
Aug 11, 2026
Merged

feat(render): render an arbitrary frame range, with the segment's own audio#169
LeadcodeDev merged 1 commit into
mainfrom
feat/frame-range

Conversation

@LeadcodeDev

Copy link
Copy Markdown
Owner

Closes the Critical gap "frame-range rendering and concatenable segments" from the re-scored Remotion differential. It is the single lock between today's state and the whole distributed/serverless axis, which is scored XL and which several High-value gaps depend on strictly.

The internals were already close

build_frame_tasks produces the complete ordered task list, render_frame_task renders one in isolation with no cross-frame state, and --frame already indexed into it. Rendering an arbitrary subset was, in principle, available. What was missing sat downstream.

What this adds

--frames a-b — inclusive, 0-indexed. Malformed input fails at the clap layer; out-of-range fails against the scenario's real total, naming both:

$ rustmotion render -f examples/demo.json -o x.mp4 --frames 10-5
error: invalid value '10-5' for '--frames <START-END>': start (10) must be <= end (5)
$ rustmotion render -f examples/demo.json -o x.mp4 --frames 99999-100000
Error: Frame range 99999-100000 is out of range (total frames: 90)

Mutually exclusive with --frame and --watch. png-seq/gif/raw refuse explicitly rather than silently ignoring the range.

Audio offset — the real hazard. This is why a naive --frames would have been worse than none at all. mix_audio_tracks had no offset parameter: every segment would have received the audio from the top of the scenario. The video would have cut cleanly while the sound was wrong, with nothing to signal it.

Red phase, measured: without the offset, 99.9% of a second segment's samples were wrong (229176 of 229320 bytes).

mix_audio_tracks_segment translates each sample into absolute scenario time and reprojects it into the segment's buffer. The scenario's total duration stays a separate parameter from the segment's, so fades and unbounded track ends stay anchored to the whole scenario rather than to a segment edge.

rustmotion concat joins segments through ffmpeg's concat demuxer with -c copy.

Why not join the bitstreams

Raw Annex-B joining requires every segment boundary to land on an independently decodable keyframe. That holds on the native openh264 path — which forces an intra frame on every frame, a pre-existing property unrelated to this change — but not on the default path, where ffmpeg is auto-detected and libx264/libx265 manage their own GOP structure with no per-frame control. Joining bitstreams there would produce a silently corrupt or undecodable stream at some cuts.

The concat demuxer sidesteps it: it trusts each segment's container and touches no pixels. Assumed implication: rustmotion concat depends hard on ffmpeg being on PATH, with no software fallback — consistent with the default render path already depending on it.

Verification

The end-to-end check, reproduced with the real binary rather than only in tests:

whole.mp4 frames=90 duration=3.000000
joined.mp4 frames=90 duration=3.000000

And the segment mixer is byte-for-byte identical to the whole-scenario mix once its segments are concatenated.

Fixed along the way

A pre-existing race in encode_with_ffmpeg_hw: the audio scratch directory was named by PID alone, so concurrent encodes within one process shared it and one call's cleanup deleted a directory another was still writing to. Surfaced by adding four audio renders to the test suite; it now carries an atomic counter alongside the PID.

What distributed rendering still needs on top of this

  1. Keyframe control at segment boundaries, if bitstream joining ever becomes preferable to remuxing.
  2. Automatic splitting into N balanced segments — today the caller computes each a-b from the total (rustmotion info reports it).
  3. Decoded-PCM caching across segments of one jobmix_audio_tracks_segment re-decodes each source track in full per segment. Correct, but N segments means N full decodes.
  4. Cross-segment validation before concatconcat lets ffmpeg fail on codec/resolution mismatch rather than diagnosing it first.
  5. Streaming/upload rather than gathering every segment locally before joining.

… audio
Closes the Critical gap that gates the whole distributed/serverless axis:
until now a scenario could only be rendered whole, or one frame at a time.
The internals were already close. `build_frame_tasks` produces the complete
ordered task list, `render_frame_task` renders one in isolation, and
`--frame` already indexed into it. What was missing sat downstream.
- `--frames a-b` (inclusive, 0-indexed). Malformed input fails at the clap
layer; out-of-range fails against the scenario's real total, naming both
the range and that total. Mutually exclusive with `--frame` and `--watch`.
`png-seq`/`gif`/`raw` refuse explicitly rather than silently ignoring the
range — those encoders live outside this change's file scope.
- Audio was the real hazard, and the reason a naive `--frames` would have
been worse than none. `mix_audio_tracks` had no offset parameter: every
segment would have received the audio from the top of the scenario, so
the video would have cut cleanly while the sound was wrong, with nothing
to signal it. `mix_audio_tracks_segment` translates each sample into
absolute scenario time and reprojects it into the segment's buffer. The
scenario's total duration stays a separate parameter from the segment's,
so fades and unbounded track ends remain anchored to the whole scenario
rather than to a segment edge. Measured on the red phase: without the
offset, 99.9% of a second segment's samples were wrong.
- `rustmotion concat` joins segments through ffmpeg's concat demuxer with
`-c copy`. Raw Annex-B bitstream joining was rejected deliberately: it
requires every segment boundary to land on an independently decodable
keyframe, which holds on the native openh264 path but *not* on the
default ffmpeg path, where libx264 manages its own GOP structure. Joining
bitstreams there would produce a silently corrupt stream at some cuts.
Verified end to end, not just in unit tests: the same scenario rendered
whole and as two concatenated segments gives 90 frames and 3.000000s either
way. The segment mixer is byte-for-byte identical to the whole-scenario mix
when its segments are concatenated.
Also fixes a pre-existing race found while testing this: the ffmpeg audio
scratch directory was named by PID alone, so concurrent encodes in one
process shared it and one call's cleanup deleted a directory another was
still writing to. It now carries an atomic counter as well.
@LeadcodeDevLeadcodeDev added the enhancement New feature or request label Aug 10, 2026
@LeadcodeDevLeadcodeDev self-assigned this Aug 10, 2026
@LeadcodeDev
LeadcodeDev merged commit a0fd570 into mainAug 11, 2026
3 checks passed
@LeadcodeDev
LeadcodeDev deleted the feat/frame-range branch August 11, 2026 07:02
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@LeadcodeDev