Search indexer perf: dead code + name mismatches + per-M2M batching

[ajslater]

Summary

Implementation of tasks/search-perf/00-plan.md (PR #625, still in flight). Two commits.

Commit 1 — Dead code + name mismatches + small wins (fc0498a2)

F1 — drop _prefetch_related_fts_query. Empirically verified that
prefetch_related() chained to .values() doesn't fire prefetches:

>>> qs = Comic.objects.prefetch_related("characters").values("pk")
>>> with CaptureQueriesContext(connection) as ctx:
...     list(qs)
>>> len(ctx.captured_queries)
1   # only the main SELECT — zero prefetch queries

The 11 prefetch declarations (and the misleading "1000 sqlite query
depth" comment) were doing zero query work.

F2 — reconcile column-name mismatches between annotation aliases
and consumer keys. _M2M_FTS_RELS produced annotations under
fts_credits__person, fts_identifiers__source,
fts_story_arc_numbers__story_arc, but prepare.py:_COMIC_KEYS read
fts_credits / fts_sources / fts_story_arcs. The dict
comprehension's if comic.get(key) filter silently dropped the
mismatched data — three GROUP_CONCAT aggregations ran, results were
discarded, and the resulting FTS columns were always empty for any
sync-built entry.

Search by credit-person name / identifier-source / story-arc title
silently returned no hits before this fix. Behavior change: those
searches now return real results.

Fix: split the rel definition into a name → path map.
_M2M_FTS_REL_MAP keeps the alias short (matches _COMIC_KEYS) but
preserves the FK-traversal path for GROUP_CONCAT. Same SQL generated;
just the alias name changes.

F4 — hoist ComicFTS.exists() out of the operate loop. After
iteration 1 lands, ComicFTS has rows for the rest of the run; the
per-iteration .exists() SELECT was wasted.

F5 — time() → monotonic() for elapsed_time reporting.
Wall-clock jumps (NTP / DST / manual adjustment) skew
time() - start_time. Same fix shape as PR #623 / #624 / #625 in
other librarian modules.

Commit 2 — Per-M2M batched queries (the headline) (1b00c9dd)

The "megaquery" (1 SELECT with 10 GROUP_CONCAT aggregates + 10 LEFT
JOIN'd FK columns + GROUP BY) materialized a cartesian product per
batch. For a comic with 5 chars × 3 credits × 2 genres × 4 tags, the
intermediate temp table before GROUP BY had 5 × 3 × 2 × 4 × ...
rows. SQLite sorted each GROUP_CONCAT for DISTINCT before collapsing.

Apply the OPDS metadata batching pattern:

	per batch
Before	1 megaquery, 30+ JOINs, cartesian product
After	12 small queries (1 FK + 10 M2M + 1 universes), each walks one relation's index

Each per-M2M query is a single LEFT JOIN + GROUP BY comic_id —
no cartesian product across the 10 M2Ms. Cost goes from
O(product of M2M counts × batch_size) to O(sum of M2M-table-rows per relation).

Three new helpers:

• _build_fk_fts_rows(pks) — comic attrs + simple FK names. One
  SELECT, no M2M, no GROUP BY. Returns list of dicts keyed by id.
• _build_m2m_fts_dict(pks, target_path) — per-M2M
  GROUP_CONCAT. Returns dict[pk, comma_string].
• _build_universes_fts_dict(pks) — special case for universes
  (Concat(designation_aggregate, ",", name_aggregate)).

The batch loop builds the FK rows + 10 M2M dicts + universes dict,
then stitches per-pk dicts in Python before passing to
prepare_sync_fts_entry. Iteration order preserved via order_by("pk")
on the FK query.

Drive-by cleanup: drop _select_related_fts_query (defined but never
called — dead code) and the now-unused _M2M_FTS_ANNOTATIONS
constant.

Test plan

• make lint-python clean (ruff + format + basedpyright +
  vulture + complexipy + codespell).
• make test-python clean (26 tests pass).
• make lint-frontend / make test-frontend clean.
• On a populated install, fire SearchIndexSyncTask(rebuild=True)
  and watch wall time — should drop noticeably for richly-tagged
  libraries.
• After F2: search for a known credit-person name / story-arc
  title / identifier-source URN returns hits. Before F2: those
  searches returned zero.
• Spot-check the resulting FTS table for a sync-built entry —
  credits / sources / story_arcs columns should now contain
  real data instead of empty strings.

References

• Plan: tasks/search-perf/00-plan.md (PR Search indexer perf: plan #625)
• codex/views/opds/metadata.py:get_m2m_objects_by_comic — the
  per-M2M UNION batching pattern this PR mirrors
• PR Serializers perf 03: OPDS v2 batching + link/unused cleanup #615 (OPDS v2 batching) — same shape applied to OPDS feeds
• PR Threads + worker: gate type-only imports + monotonic clock #623 / Status controller perf: monotonic clock + drop wasted SELECT + gate type-only imports #624 / Search indexer perf: plan #625 — the time() → monotonic() fix
  applied to other librarian modules

🤖 Generated with Claude Code