Skip to content

perf: start the load at song:loading, before the highway is drawn (fixes the 698ms video freeze) - #40

Merged
byrongamatos merged 1 commit into
mainfrom
perf/preload-stems-graph
Jul 14, 2026
Merged

byrongamatos merged 1 commit into
mainfrom
perf/preload-stems-graph

Conversation

@byrongamatos

@byrongamatos byrongamatos commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Building the graph hands every stem's decoded PCM to the audio worklet — which means copying the whole song. For a 4-minute 6-stem pack that is over half a gigabyte of memcpy, in one frame, on the main thread.

It ran on the highway's WS ready — with the player already on screen — and it froze the picture. Measured on a real load: a 698 ms frame, right as the song-credits card appeared, with the venue video visibly stopping. That is the "it's the video that pauses/drops frames the moment the author appears" report.

The reason it could not start sooner: the stem list only ever arrived with that WS ready. Core now publishes the same list at GET /api/song/{f}?stems=1 (got-feedBack/feedBack#972), so the whole load — fetch, decode, graph build — starts on song:loading instead. Nothing about the work changes; only when.

Verified on the real build

The ~700 ms stall moved to before the highway is up, behind the loading overlay where a stalled frame costs nothing:

#98    709.6 ms   <- the PCM copy, BEFORE the highway
#102              song:loaded  (WS ready / highway up)
#114   168.6 ms   credits
worst frame after the highway appears
before 698 ms
after 169 ms

Safe by construction

  • The highway's list still wins the moment it arrives; the preload is only a stand-in before it. Core builds both from one shared helper and pins them byte-identical, so the signature matches and the ready path skips a second identical build rather than rebuilding.
  • Generation-counted: an in-flight preload for a previous song cannot overtake a newer one, or it would build the wrong song's graph over the current one.
  • Entirely best-effort: an offline core, a non-200, an older core that does not know ?stems=1, or a stem-less pack all fall through to the old ready-driven path.

Tests: 6 new (all failing against the pre-fix source), suite 53/53.

Summary by CodeRabbit

  • Performance Improvements

    • Song stem metadata now begins loading earlier, helping playback initialize faster.
    • Playback can proceed using preloaded metadata when live song information is not yet available.
  • Reliability

    • Prevented outdated preload requests from affecting newly selected songs.
    • Added graceful fallback behavior when stem metadata cannot be retrieved.
    • Live song information takes precedence when it becomes available.
  • Tests

    • Added coverage for preload timing, fallback behavior, request handling, and song-matching safeguards.

Building the graph hands every stem's decoded PCM to the audio worklet — which
means copying the WHOLE SONG. For a 4-minute 6-stem pack that is over half a
GIGABYTE of memcpy, in one frame, on the main thread.

It ran on the highway's WS `ready`, i.e. with the player already on screen, and
it froze the picture. Measured on a real load: a 698 ms frame, right as the
song-credits card appeared, with the venue video visibly stopping ("it's the
video that pauses/drops frames the moment the author appears").

The reason it could not start sooner is that the stem list only ever arrived with
that WS `ready`. Core now publishes the same list at GET /api/song/{f}?stems=1
(feedBack#…), so the whole load — fetch, decode, graph build — starts on
`song:loading` instead. Nothing about the work changes; only WHEN.

Verified on the real build. The ~700 ms stall moved to BEFORE the highway is up,
behind the loading overlay where a stalled frame costs nothing:

    before:  the 698 ms frame lands AT the credits, venue on screen
    after:   709 ms at frame #98; song:loaded (highway up) is frame #102
             worst frame after the highway appears: 698 ms -> 169 ms

Safe by construction:
  - the highway's list still wins the moment it arrives; the preload is only a
    stand-in before it. Core builds both from one shared helper and pins them as
    byte-identical, so the signature matches and the ready path skips a second,
    identical build rather than rebuilding.
  - generation-counted: an in-flight preload for a previous song cannot overtake
    a newer one, or it would build the wrong song's graph over the current one.
  - entirely best-effort: an offline core, a non-200, an older core that does not
    know ?stems=1, or a stem-less pack all fall through to the old ready-driven
    path.

Tests: 6 new (all failing against the pre-fix source), suite 53/53.
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Playback loading now preloads stem metadata from the REST API, stores it in shared state with generation protection, and allows initialization to use matching preload data until live highway metadata becomes available.

Changes

Stem metadata preload

Layer / File(s) Summary
Preload state and metadata resolution
src/state.js, src/main.js
Shared preload fields, REST stem fetching, generation checks, filename matching, and live-data precedence were added.
Playback initialization and validation
src/main.js, tests/preload.test.mjs
Playback loading starts the preload, initialization uses currentSongInfo(), and tests cover ordering, fallback, stale requests, and metadata precedence.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Playback
  participant Loading
  participant Preload
  participant REST
  participant Init
  participant Highway
  Playback->>Loading: emit song loading
  Loading->>Preload: preload current filename
  Preload->>REST: request stems metadata
  REST-->>Preload: return stem metadata
  Preload-->>Init: provide matching preload info
  Init->>Highway: check live song info
  Highway-->>Init: return live metadata when ready
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly reflects the main change: moving loading to song:loading to reduce the visible freeze.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/preload-stems-graph

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/main.js (1)

972-972: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Preload fetch has no abort signal.

Unlike the other fetches in this file (probe, loadStems, loadFullMix), this fetch isn't tied to an AbortController. A superseded preload keeps downloading to completion even though its result will be discarded by the gen/filename check — harmless for correctness, just a wasted request on rapid song switches.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/main.js` at line 972, Update the preload fetch in the surrounding preload
flow to accept and pass the same AbortController signal used by probe,
loadStems, and loadFullMix. Ensure superseded preloads are aborted while
preserving the existing gen/filename result validation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/main.js`:
- Line 972: Update the preload fetch in the surrounding preload flow to accept
and pass the same AbortController signal used by probe, loadStems, and
loadFullMix. Ensure superseded preloads are aborted while preserving the
existing gen/filename result validation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e996b648-b42f-4030-91ca-f1afb6e547ad

📥 Commits

Reviewing files that changed from the base of the PR and between 1a1fc6e and 0ef885a.

📒 Files selected for processing (3)
  • src/main.js
  • src/state.js
  • tests/preload.test.mjs

@byrongamatos
byrongamatos merged commit 8ac42e9 into main Jul 14, 2026
3 checks passed
carochacs added a commit to get-flashbacks/feedBack-plugin-stems that referenced this pull request Sep 11, 2026
* fix: recover player when initial stems stream pump fails

setupStreaming() reports success once the graph/worklet are built, but
runPump() keeps prefilling asynchronously afterward. If that prefill
rejected (a stem reader dying, a dropped fetch), the failure was only
logged: core <audio> stayed paused (silenced for the takeover) with
nothing resuming it, so the play request stayed pending and the player
appeared to be loading/playing with no audible output.

Route a genuine pump failure (not a superseded seek, not an
intentional stop) through the same recovery already used for a
synchronous setupStreaming()/buildGraphFromBuffers() failure: tear the
takeover down, clear the reroute guard, and resume core <audio> if
playback was requested.

Fixes got-feedBack#40

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor: consolidate stems takeover-failure recovery into one helper

Codacy review on PR #6 flagged that the recovery sequence (teardown +
clear reroute guard + resume core <audio>) was duplicated across four
failure sites in onSongReady/onStreamFailure, three of them also
calling hideOverlay() redundantly since teardown() already hides it.

Extract _revertToCoreAudio(shouldResume) (teardown + reroute-guard
clear + conditional resume) and _resumeCoreAudio() (resume only, for
the one site where buildGraphFromBuffers already tore down
internally) and use them at all four sites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant