fix(api): stop profile/dashboard track queries from seq-scanning tracks (perf regression from #932)

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code

[raymondjacobson]

Incident hotfix

After #932 deployed, GET /users/:id/tracks (and the dashboard monthly-listens query) started sequential-scanning the entire tracks table on every request, melting the DB on a top-QPS endpoint.

Root cause

#932 merged accepted-collaborator tracks into the profile query as:

WHERE (t.owner_id= $1ORt.track_idIN (SELECT track_id FROM track_collaborators WHERE collaborator_user_id = $1AND status ='accepted'))

That OR across two different access paths (an indexed column on tracks vs. a semi-join subquery) defeats the tracks(owner_id) index. With the subquery's poor row estimate, the planner falls back to a seq scan of tracks (+ the aggregate_* joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).

Fix

Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering (collaborator_user_id, status, track_id) index that returns nothing for ~every user — then:

• No collaborations (≈everyone): run the original owner-only query, byte-identical to pre-feat(api): collaborative tracks — notifications, embed, profile + dashboard merge #932 (same plan, same perf). The extra lookup is one cheap index probe.
• Has collaborations (rare): widen via an explicit t.track_id = ANY($ids) array, which the planner handles with a bitmap OR of the owner_id index and the track_id PK — not a correlated subquery.

Same treatment applied to v1_users_listen_counts_monthly.

Correctness

Behavior is unchanged. Passing: TestGetUserTracks (no-collab path), TestAcceptedCollaborationAppearsOnProfile (merge still works), TestPendingCollaborationHiddenFromProfile, TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.

Note

This is the perf risk I'd flagged in the #932 review (the OR … IN pattern). If you need relief before this merges, reverting just v1_users_tracks.go + v1_users_listen_counts_monthly.go to their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.

🤖 Generated with Claude Code