Discussion: split `server.py` into routers + lib modules?

[byrongamatos]

Restored from slopsmith/slopsmith#172 — original issue, opened by @topkoa on 2026-05-04.
[restored-from: slopsmith/slopsmith#172]

server.py is 2985 lines today and growing. It's been working fine and is grep-friendly as one file, but I wanted to float whether a structural split would pay off before the next round of features lands. Opening this for discussion rather than a PR — happy to abandon if folks prefer the status quo.

What's in server.py today

A rough map of the chunks I see, by line range:

Lines	Subsystem
53–220	demo-mode middleware + janitor hook registry
222–660	MetadataDB class (~440 lines, SQLite schema + queries)
661–970	DLC discovery + _extract_meta_* + background scanner
966–1225	startup/shutdown/rescan endpoints
1245–1390	library / artists / stats / favorites / loops endpoints
1392–2080	settings load/save + Settings export/import (allowlist, encode, decode, atomic write)
2084–2215	diagnostics export/preview/hardware endpoints
2216–2333	retune WebSocket
2334–2510	song meta / art / get_song_info / sloppak file serve
2545–2970	highway WebSocket (~425 lines, the largest single block)

Why split

• Navigation. Jumping between the highway WS and MetadataDB schema means scrolling 2000+ lines or relying on bookmarks. Splitting puts each subsystem under one screen-height.
• Merge conflict surface. Settings export/import, diagnostics, and the highway WS all get touched in the same release cycles. They live ~700 lines apart in one file, but git sees them as the same file — concurrent PRs collide on imports and the FastAPI route block.
• Test ergonomics. Most of MetadataDB is plain SQLite. Today tests must import server, which drags in FastAPI, the startup scan, the WebSocket handlers, and every helper. A lib/db.py could be imported standalone.
• Plugin-context surface is implicit. The context dict passed to plugin setup() is built inline and silently picks up whatever module-level names exist. Pulling that build into one place would make the plugin contract reviewable.

Why NOT split (or be careful)

• app is global. Every endpoint decorator binds to it. Splitting requires an APIRouter per module and app.include_router(...) in server.py. Mechanical, but touches every endpoint.
• Shared module-level state. Things like _extract_lock, meta_db, _extract_meta_for_file, the startup-status dict, and the demo-mode flag are read by handlers and helpers in different chunks. Splits need a single home for that state (e.g. lib/state.py) or explicit DI through app.state — and getting it wrong means subtle bugs where two modules each get their own copy of a "shared" lock.
• Plugin contract is callable identity. Plugin context[\"extract_meta\"], context[\"meta_db\"], etc. are passed as live references into already-loaded plugins. Refactoring must preserve the exact callables — moving them to a new module is fine, but renaming or wrapping them breaks third-party plugins. We'd want a "plugin context unchanged" assertion in CI.
• lib/ convention is flat imports. No __init__.py, pythonpath = [\".\", \"lib\"]. A split should match that — flat lib/db.py, lib/scan.py, lib/highway_ws.py rather than a nested lib/server/ package.
• Reviewer cost. A split PR is a giant move-only diff that hides any actual logic change inside it. Would need to be done as a strict no-op move with git mv-style line preservation, and any behavior change rides a separate follow-up.
• Search habits. grep on one file is muscle memory for a lot of contributors. A split adds a tiny mental tax on "where does X live now?"

A concrete target shape (for discussion only)

server.py                     FastAPI app, lifespan, router includes (~300 lines)
lib/db.py                     MetadataDB
lib/scan.py                   _background_scan, _extract_meta_*, _periodic_rescan
lib/state.py                  shared module-level globals (locks, status dicts)
routers/library.py            library / artists / stats / favorites / loops
routers/settings.py           settings + export/import + helpers
routers/diagnostics.py        diagnostics endpoints
routers/song.py               song meta/art/get_song_info/serve_sloppak_file/audio
ws/retune.py                  retune WS
ws/highway.py                 highway WS — biggest single win

This is a sketch, not a proposal. Boundaries can shift (scan could merge into db; state may not be needed if we lean on app.state).

Open questions for the discussion

1. Is the current single-file structure deliberate (e.g. for vibe-coding ergonomics, distribution simplicity, deploy-time grep-ability) or just "hasn't needed splitting yet"?
2. Would a phased split — start with one or two highest-value extractions (MetadataDB, highway WS) and see how it feels — be preferable to a single big-bang refactor?
3. Any constraint I'm missing? E.g. plugin-context callable identity, demo-mode middleware ordering, startup-event registration order.

If reactions are positive, I'd plan to do the split as a sequence of strict move-only PRs, each rebaseable, with no behavior changes — and any behavior changes go in separate follow-ups. If reactions are "leave it alone, the file is fine", happy to close this.

[byrongamatos]

@byrongamatos commented on 2026-05-04:

Good analysis — the proposed shape is solid and the phased approach is the right call. A few thoughts:

Hold the full split until merge pain is actually felt. The strongest argument for the refactor is concurrent PRs colliding on the same file, and that's not happening yet. The reviewer burden and rebase risk of a big-bang split aren't justified by preemptive ergonomics alone.

But two extractions are worth doing now while they're still cheap:

• MetadataDB → lib/db.py: zero shared-state risk, clear boundary, immediate payoff in test ergonomics (tests can import it without dragging in FastAPI + the scanner + every WS handler). This is the easiest one and the benefit is concrete.
• Highway WS → lib/highway_ws.py: the longest single block and the most actively touched by plugin contributors. Isolating it now, while the boundaries are still readable, is much cheaper than doing it at 5000 lines with more state entangled.

The rest of server.py (settings/export, library endpoints, diagnostics) can stay put until there's an actual reason to move it.

On lib/state.py: I'd skip this entirely. Moving shared mutable globals to a new module doesn't reduce coupling — it just relocates it. If the state ever becomes a problem, app.state + explicit DI through routers is the right direction, but that's a separate conversation from a move-only refactor.

On directory structure: keep it flat to match the existing lib/ convention — lib/db.py, lib/highway_ws.py rather than introducing routers/ and ws/ subdirectories alongside lib/. New source tree roots add mental overhead without much benefit at this scale.

Plugin context safety net: before any extraction lands, a tests/test_plugin_context_contract.py that asserts the expected callables are present by name would give a useful safety net — catches any accidental rename or wrapper that breaks third-party plugins.

Happy to help with either of the two extractions if you want to take them on.