Render harness gives every (size, mode) a fresh empty MockCacheManager — baseball-scoreboard takes 840s where 72s would do, and the cache-hit path is never tested

[ChuckBuilds]

Summary

_instantiate builds a fresh MockCacheManager() for every (size, mode) render, and MockCacheManager is a per-instance in-memory dict. So a plugin that caches network results gets an empty cache on every render and re-fetches everything from scratch, N times over.

For plugins that fetch per game or per player, the cost compounds badly. baseball-scoreboard at a single size goes from 18s to 840s — 93s per render — purely from re-fetching data the first render already had.

It also means the harness only ever exercises the cache-miss path. The cache-hit path, which is what a running rig executes almost all of the time, is never rendered.

Measurements

baseball-scoreboard 1.40.1, one panel size (--sizes 128x64), core v3.3.0-4-g0730d952, rig otherwise idle apart from the display service:

config	renders	wall clock	per render
default (MLB, default options)	3	2s	0.7s
MLB only + all display_options	3	8s	2.7s
3 leagues, default options	9	18s	2.0s
3 leagues + all display_options	9	840s	93s

The multiplicative estimate for the last row is ~72s (18s × the 4× that all-options costs on MLB). Actual is 11.7× that. The extra is re-fetching: with show_odds, show_player_card, show_pitcher_batter and show_last_play all on, the plugin issues per-game and per-player ESPN calls, and none of them survive into the next render.

At the full 8 sizes this variant exceeded a 900s timeout, which is what first drew my attention.

Mechanism

src/plugin_system/testing/harness.py:117-140:

def _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, display_manager):
 cache_manager = MockCacheManager()
 for key, value in (mock_data or {}).items():
 cache_manager.set(key, value)
 ...

_render_size calls this once per mode (plus a probe), and render_plugin_matrix calls _render_size once per size — so a plugin with 3 modes at 8 sizes gets 32 independent empty caches.

src/plugin_system/testing/mocks.py:62-78:

class MockCacheManager:
 def __init__(self):
 self._cache: Dict[str, Any] = {}
 ...
 self.cache_dir = tempfile.mkdtemp(prefix="ledmatrix-mock-cache-")

Nothing is shared between instances, and the temp dirs are removed by a finalizer. On this rig every /tmp/ledmatrix-mock-cache-* directory is empty, which is what led me here.

Why it matters

1. CI cost and flakiness. Every render is network-bound against ESPN. A plugin exercising several leagues with rich display options can take a quarter of an hour at one size.
2. The tested path isn't the production path. Production runs with a warm CacheManager and honours odds_update_interval (3600s default). The harness renders the cold-start path exclusively, so a caching regression — stale reads, key collisions, a cache that never hits — cannot be caught here.
3. It interacts with load_config_defaults ignores nested schema defaults — 2,386 defaults across 37 of 44 plugins never reach the render harness #531. Nested defaults not loading means most sports plugins render with one league; the moment that's fixed and several leagues switch on, this cost multiplies across the fleet.

Suggested fix

Share one MockCacheManager across all renders of a plugin within a render_plugin_matrix call — thread it in from render_plugin_matrix rather than constructing it inside _instantiate:

def render_plugin_matrix(...):
 cache_manager = MockCacheManager()
 for key, value in (mock_data or {}).items():
 cache_manager.set(key, value)
 ... # pass it down to _render_size / _instantiate

That keeps per-render isolation of the display manager (which is what the bounds checking needs) while letting fetched data be reused, and it makes the second and later renders exercise the cache-hit path.

If per-render isolation of the cache is deliberate for some tests, an opt-in would do — harness.json: {"share_cache": true} — but the default that costs 840s where 72s would do seems worth flipping.

Not a plugin bug

Worth stating explicitly since the headline number looks alarming: I first took the 840s for a baseball-scoreboard performance regression. It isn't. The plugin's per-game fetching is documented in its own source (sports.py:2025-2031 notes that the favourites narrowing "applies only when show_favorite_teams_only is set AND favourites are configured — neither is the default"), and with a real cache those fetches are amortised over an hour. The blow-up is the harness discarding the cache between renders.

Environment

LEDMatrix v3.3.0-4-g0730d952, Raspberry Pi (3 cores), 256x64 panel, 44 first-party plugins installed.

[ChuckBuilds]

One more data point from the same ladder, which rules out the obvious "just narrow the fetch" response:

config	renders	wall clock
3 leagues + all display_options	9	840s
3 leagues + all display_options + show_favorite_teams_only: true, show_all_live: true	9	840s

Identical. Turning the favourites filter on changes nothing, because no favorite_teams are configured — which is exactly what baseball-scoreboard/sports.py:2025-2031 already says:

# applies only when show_favorite_teams_only is set AND favourites
# are configured -- neither is the default -- so in the usual case
# nothing narrowed it. Each request is a separate ESPN call on a Pi
# that is also driving the panel.
if self.show_odds:
 for game in team_games:
 self._fetch_odds(game)

So the measurement confirms that comment precisely, and the cost really is the repeated cold-start fetching described above rather than anything a plugin-side filter can reduce.