Skip to content

feat(editor): studio stems play — the multi-source stem engine - #293

Merged
byrongamatos merged 1 commit into
mainfrom
feat/editor-stem-engine
Jul 16, 2026
Merged

byrongamatos merged 1 commit into
mainfrom
feat/editor-stem-engine

Conversation

@ChrisBeWithYou

@ChrisBeWithYou ChrisBeWithYou commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

What this is

PR 5 of 5 — the final #285 salvage slice (stacked on #292#291#290#289 → main). Makes studio stems audible: they play alongside the master, sample-aligned, each with its own mute/solo/fader on its Tracks row.

Design: additive, not master-replacing

Unlike #285 (which unified master+stems into one scheduler and aliased S.audioSource), this keeps the master path completely untouchedS.audioSource → _refGain, the audition MediaElement, A/B compare — and adds stems as a parallel scheduler hooked into the same start/stop/restart points. A stem-engine fault can't take the recording down with it. Lower risk, smaller blast radius, and the tricky master paths (audition, first-play fade, loop) keep working unchanged.

How it works

  • DecodestemAudioCache (per-source {url, buffer, peaks}), syncStemAudio decodes all live stems in parallel (generation-guarded; one unavailable stem never blocks the rest), resetStemAudioCache on song boundary. Triggered from loadCDLC, the create path, and every /stem-op adopt (import/rename/reorder/remove).
  • Schedule_startStemSources creates a BufferSource per cached stem at the same preRoll-shifted anchor as the master, each placed from its own S.audioShift + source.offset via the shared _audioBufferStartPure. That shared placement math is the alignment contract — stems stay locked to the master through seeks (_restartPlaybackAt), loop wraps, and audio-shift. _stopStemSources is wired into restart/stop/teardown.
  • Mix — each stem's gain connects to _refGain (the transparent path, never the guide limiter), ramped by applyStemMix off host.partStripState('audio:<id>') — the same S.partMix store and whole-map solo rule the synth parts use. One mixer, two bands: soloing a stem silences unsoloed synth parts and vice-versa, while the master reference stays audible.
  • Capabilityhost.stemMixChanged is now wired (its presence is what flips stemMixerAvailable() true). That lights up: Solo-my-source (reconciled to write S.partMix['audio:<id>'] — an exclusive isolate over the audio band — instead of the dead S.stemMix stub), the PR-5 audio-row strips (stems get M/S/fader; the master mix stays strip-less since it's the ungated reference), and per-stem lane waveforms (audioStemWaveform from the decoded buffer).

Scoped deliberately

Unity fader ceiling (no +6 dB) and no live meters — both are additive polish (_mixerGainForFaderPure log mapping, Analyser taps, the rAF meter loop) that change the vol contract; a clean follow-up. Known limitation: at audition speed < 1 the master reroutes to a pitch-preserving MediaElement and the sample-accurate BufferSource path (stems included) is silenced — stems resume at 100%. Documented in the changelog and code.

Tests

  • tests/stem_engine.test.mjs (new, 4): the mixer lists audio:<id> strips (band order, removals honored), a stem strip reads S.partMix, the whole-map solo rule spans stems + synth parts, and the per-source placement math (each stem from its own shift+offset; skip-when-ended; delay-when-negative).
  • tests/stem_tracks.test.mjs: the three Solo-my-source cases reconciled to the partMix['audio:<id>'] model (incl. a new assertion that a synth-part solo is left alone — this verb owns the audio band only).
  • tests/audition_clock.test.mjs: injects the new _stopStemSources/_startStemSources stubs into the sliced _startAudioSourceAtCursor env.

Gates: 183 JS suites, pytest 291 / 2 skipped, lint 0 errors / 3 baseline warnings. All touched modules load without an import cycle.

⚠️ Interactive desktop verification pending (as with #289#292) — actual multi-stem playback/sync is not unit-testable. A desktop pass with a real stem session is the close condition; a build is being produced for exactly that.

The salvage is complete

With this, every feature from #285 has been re-cut onto the current main across #289#293. #285 can be closed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EBQCHCNA81E9tHmSDHSe2Q

Summary by CodeRabbit

  • New Features
    • Added Studio stems playback alongside the master recording, with per-stem mute/solo/volume controls in the mixer and track views.
    • Added “Solo my source track” isolation plus per-stem waveform lanes, and extended mixer controls to audio rows in the session header.
  • Bug Fixes
    • Improved sample-accurate stem synchronization across loading, creation, and playback lifecycle transitions, including resuming when audition speed returns to 100%.
    • Ensured solo/mute state is cleared correctly when stems are removed.
  • Tests
    • Added automated coverage for stem mixing, scheduling, and solo behavior.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Studio stems now synchronize with master playback, use per-stem mixer controls, support source-track soloing, expose waveform data, and reset during session loads and imports. Playback lifecycle handling includes normal-speed scheduling and cleanup for slow audition mode.

Changes

Studio stem playback

Layer / File(s) Summary
Stem mixer model and controls
src/mixer-panel.js, src/track-session.js, src/stem-tracks.js, tests/stem_tracks.test.mjs
Stem strips use audio:<id> keys, receive mute/solo/volume controls, and support exclusive “Solo my source track” behavior.
Stem decoding and playback engine
src/audio.js
Stem audio is decoded, cached, waveform-processed, mixed, scheduled against the master clock, and stopped or reset through playback teardown.
Loading, host hooks, and waveform integration
src/create.js, src/file-ops.js, src/main.js
Imports and CDLC loads reset and synchronize stem audio; host hooks refresh stem state and non-master waveform data.
Playback and feature validation
tests/stem_engine.test.mjs, tests/audition_clock.test.mjs, CHANGELOG.md
Tests cover stem timing, mixing, pruning, and audition-rate behavior, with the changelog documenting the feature.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant MixerPanel
  participant MainHooks
  participant AudioEngine
  participant MasterRecording
  User->>MixerPanel: change stem mute/solo/volume
  MixerPanel->>MainHooks: trigger partMixChanged or stemMixChanged
  MainHooks->>AudioEngine: applyStemMix
  User->>MasterRecording: start playback
  MasterRecording->>AudioEngine: establish transport timing
  AudioEngine->>AudioEngine: decode and schedule aligned stems
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a multi-source studio stem playback engine in the editor.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/editor-stem-engine

Comment @coderabbitai help to get the list of available commands.

@ChrisBeWithYou
ChrisBeWithYou force-pushed the feat/editor-tracks-ui branch from 76bf4ae to 2243b39 Compare July 16, 2026 03:54
@ChrisBeWithYou
ChrisBeWithYou force-pushed the feat/editor-stem-engine branch from 0993e40 to 8724902 Compare July 16, 2026 03:54
@ChrisBeWithYou
ChrisBeWithYou force-pushed the feat/editor-tracks-ui branch from 2243b39 to 977386b Compare July 16, 2026 03:57
@ChrisBeWithYou
ChrisBeWithYou force-pushed the feat/editor-stem-engine branch 2 times, most recently from 31c7381 to 2c59788 Compare July 16, 2026 04:11
byrongamatos added a commit that referenced this pull request Jul 16, 2026
Two faults in the multi-source stem engine, both reachable through normal
remove/rename of a source track while the whole-map solo rule and real gain
nodes are live (both landed with #293):

1. Phantom solo silences the live band. Solo-my-source (and the strip S
   button) write 'audio:<id>' into S.partMix — the store the whole-map solo
   rule scans. Removing a soloed stem tombstoned the track but left its solo
   entry behind, so `_mixerAnySoloPure` still saw a solo and muted every
   LIVE stem and synth part while the soloed source no longer existed. The
   drum-delete path already prunes (delete S.partMix.drums); the audio path
   didn't.

2. A removed stem kept sounding. syncStemAudio() dropped the stale cache
   entry but only stopped the playing node inside _startStemSources, AFTER
   awaiting the new roster's decodes — so a removed/replaced stem played on
   for seconds while a new (possibly large) stem decoded.

Fix both at the roster-sync chokepoint: _pruneStaleStems() runs at the top
of syncStemAudio (before any await) and, for every id no longer live, stops
its source node, disconnects its gain, and deletes its 'audio:<id>'
partMix entry. Regression test asserts a removed stem's stale solo stops
muting the live band (fails pre-fix: no prune existed).

Not fixed (noted for follow-up): _audioTimelineDuration ignores stem buffer
length/offset, so a stem longer than the master (or pushed past its end by a
positive offset) is cut at the master-based transport end. Demucs stems share
the master's length, so this doesn't arise in the primary studio ingest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@byrongamatos

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Around line 12-20: Update the Studio stems changelog entry to document that
transport ends when the master buffer ends, so longer or positively offset stems
may be cut off; place this limitation beside the existing slow-audition note.

In `@src/audio.js`:
- Around line 1656-1660: Update the stale-solo pruning logic in the S.partMix
cleanup block to reapply the current live gains to surviving gain nodes after
deleting removed audio entries. Ensure delayed fetch or decode catch-up cannot
leave live stems at previously silenced values, while preserving the existing
removal of stale `audio:` keys.
- Around line 1607-1616: Update the stem-loading flow around stemAudioCache and
_startStemSources so a cached entry whose URL differs from source.url is removed
before fetching or decoding the replacement, and stop or invalidate any
currently playing node for that stem ID. Preserve valid same-URL cache reuse
while ensuring fetch or decode failure cannot leave the previous audio eligible
to continue playing.

In `@src/main.js`:
- Around line 532-536: Update the audioSourcesChanged callback in src/main.js
(lines 532-536) to preserve the immediate draw and schedule another draw after
syncStemAudio() settles; apply the same settled-synchronization redraw in
src/create.js (lines 2329-2330) and src/file-ops.js (lines 193-197) by
scheduling host.draw() after each import or session-load stem synchronization
completes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e31f16ae-2f36-4a02-8b27-6d1d59833011

📥 Commits

Reviewing files that changed from the base of the PR and between 864ad04 and 82d4703.

📒 Files selected for processing (11)
  • CHANGELOG.md
  • src/audio.js
  • src/create.js
  • src/file-ops.js
  • src/main.js
  • src/mixer-panel.js
  • src/stem-tracks.js
  • src/track-session.js
  • tests/audition_clock.test.mjs
  • tests/stem_engine.test.mjs
  • tests/stem_tracks.test.mjs

Comment thread CHANGELOG.md Outdated
Comment thread src/audio.js
Comment thread src/audio.js
Comment thread src/main.js
@byrongamatos

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@byrongamatos
byrongamatos changed the base branch from feat/editor-tracks-ui to main July 16, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants