Skip to content

Invalid configured model path silently falls back to the cache #149

Description

@chiruu12

ModelStore.resolve_spec_path (sdk/src/unplug/ml/store.py:310) drops an explicitly configured spec.path when it fails validation and substitutes whatever is in the local cache. No warning, no error, and ml_degraded stays False.

Repro on any machine that has run unplug-models download tiny once:

importtempfile, pathlib, osos.environ.pop("UNPLUG_MODEL_PATH", None)
fromunplug.configimportGuardConfigfromunplug.core.runtime.model_runtimeimportmerge_catalog_modelsfromunplugimportGuardd=pathlib.Path(tempfile.mkdtemp()) /"corrupt"d.mkdir()
(d/"config.json").write_text("not-json")
cfg=merge_catalog_models(
GuardConfig(active_model="tiny", require_ml=False, auto_download_model=False)
)
m=dict(cfg.models)
m["tiny"] =m["tiny"].model_copy(update={"path": str(d)})
g=Guard(config=cfg.model_copy(update={"models": m}))
print(g.ml_model_loaded) # True, expected Falseprint(g.ml_degraded) # False

It loads ~/.cache/unplug/models/tiny instead. The code path:

ifspec.pathandself.is_valid_checkpoint(Path(spec.path)):
returnspeccached=self.resolve_local_path(tierorspec.name)
ifcachedisnotNone:
returnspec.model_copy(update={"path": str(cached)})

The cache lookup is a fallback for when path is unset. It also fires when path is set and invalid.

Note the asymmetry with the branch twenty lines above: an invalid UNPLUG_MODEL_PATH logs "is set but is not a valid checkpoint ... ignoring" before falling through. An invalid configured path gets nothing.

require_ml=True does not help. The fallback loads, so require_ml is satisfied and no ModelError is raised.

Why it matters: an operator pins a vetted checkpoint and turns downloads off. A path typo or a bad mount gets them a different set of weights, with ml_model_loaded=True and ml_degraded=False. Nothing in the result says the loaded model is not the pinned one.

Two existing tests already assert the correct behaviour and fail on any machine with a populated cache:

tests/unit/ml/test_model_store.py::test_guard_survives_corrupt_checkpoint
tests/unit/ml/test_model_store.py::test_guard_require_ml_corrupt_checkpoint_raises_model_error

They pass in CI only because CI has no model cache, which is also why this went unnoticed.

Fix: when spec.path is set but invalid, mirror the env branch. Log the warning and do not substitute the cached checkpoint. Raise ModelError when require_ml=True.

Related: #107, since this is a case where degraded should be set and is not.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is neededneeds-triageNot looked at yet

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions