Another warning made visible by #206 scoping optimize()'s silencing to the call itself. Notebook 4's sweep produces a steady stream of:
scipy/interpolate/_fitpack_repro.py:993: RuntimeWarning: error. a theoretically impossible result
was found during the iteration process for finding a smoothing spline with fp = s.
probably causes : s too small.
scipy is right: s really is too small, and the search space is why.
The problem
splinediff's s is an absolute residual budget — sum_t (x[t] - spline[t])**2 <= s — so the natural value scales with both signal length and noise variance, roughly N * sigma**2. But the search space is a fixed list:
{'degree': {3, 4, 5}, 's': [0.2, 0.5, 0.75, 0.9, 1, 10], 'num_iterations': [1, 5, 10]}
bounds: {'s': (0.01, 10000.0)}Sweeping s on a fine geometric grid and taking the best objective value:
| case | natural N*sigma**2 | best s (objective) | best s (true RMSE) |
|---|
| lorenz, noise scale 1 | 4.0 | 3.49 | 3.49 |
| lorenz, noise scale 4 | 64.0 | 57.90 | 57.90 |
| sine, noise scale 1 | 4.0 | 3.49 | 3.49 |
| sine, noise scale 4 | 64.0 | 57.90 | 57.90 |
The optimum tracks N*sigma**2 closely (~0.87x), and the objective picks the same s as true RMSE does, so the surrogate is working fine. The seeds are the problem: they happen to bracket the optimum at N=400, sigma=0.1 — exactly the default config — and are entirely below it at higher noise. At noise scale 4 every seed is <= 10 while the optimum is ~58, so the search starts wholly inside the regime scipy is complaining about.
Options
- Widen the seeds geometrically, e.g.
[0.5, 2, 8, 32, 128]. One line, but still wrong for a different N. - Derive the seeds from the data — multiples of
N * sigma_hat**2, with sigma_hat from something like mad(diff(x)). Correct for any N and noise level, and keeps s a genuine hyperparameter rather than pinning it. Needs optimize() to build a data-dependent search space instead of reading the static dict, which is new plumbing but has precedent in how tvgamma is derived from cutoff frequency. - Leave it if Nelder-Mead reliably climbs out from the seeds, in which case this is only noise in the logs.
Option 2 looks right. Setting s inside splinediff itself would fix the warning but foreclose optimizing over it, so that's not the way.
Unresolved: whether Nelder-Mead actually reaches ~58 from seeds capped at 10, which decides how much of this is cosmetic versus splinediff being under-served at high noise in the benchmark.
🤖 Written by Claude Code on behalf of @pavelkomarov
Another warning made visible by #206 scoping
optimize()'s silencing to the call itself. Notebook 4's sweep produces a steady stream of:scipy is right:
sreally is too small, and the search space is why.The problem
splinediff'ssis an absolute residual budget —sum_t (x[t] - spline[t])**2 <= s— so the natural value scales with both signal length and noise variance, roughlyN * sigma**2. But the search space is a fixed list:{'degree': {3, 4, 5}, 's': [0.2, 0.5, 0.75, 0.9, 1, 10], 'num_iterations': [1, 5, 10]} bounds: {'s': (0.01, 10000.0)}Sweeping
son a fine geometric grid and taking the best objective value:N*sigma**2s(objective)s(true RMSE)The optimum tracks
N*sigma**2closely (~0.87x), and the objective picks the samesas true RMSE does, so the surrogate is working fine. The seeds are the problem: they happen to bracket the optimum at N=400, sigma=0.1 — exactly the default config — and are entirely below it at higher noise. At noise scale 4 every seed is <= 10 while the optimum is ~58, so the search starts wholly inside the regime scipy is complaining about.Options
[0.5, 2, 8, 32, 128]. One line, but still wrong for a differentN.N * sigma_hat**2, withsigma_hatfrom something likemad(diff(x)). Correct for anyNand noise level, and keepssa genuine hyperparameter rather than pinning it. Needsoptimize()to build a data-dependent search space instead of reading the static dict, which is new plumbing but has precedent in howtvgammais derived from cutoff frequency.Option 2 looks right. Setting
sinsidesplinediffitself would fix the warning but foreclose optimizing over it, so that's not the way.Unresolved: whether Nelder-Mead actually reaches ~58 from seeds capped at 10, which decides how much of this is cosmetic versus splinediff being under-served at high noise in the benchmark.
🤖 Written by Claude Code on behalf of @pavelkomarov