Skip to content

[BUG] EarlyStopping honors monitor/mode but ModelCheckpoint (and weight restore) is hardcoded to val_loss/min #425

Description

@ChrisW09

Describe the bug
EarlyStopping and ModelCheckpoint monitor different metrics, and the final weights come from the checkpoint (deeptab/models/_mixins/fit.py:481-494):

early_stop_callback=EarlyStopping(
monitor=monitor, min_delta=0.00, patience=patience, verbose=False, mode=mode
)
checkpoint_callback=ModelCheckpoint(
monitor="val_loss",
mode="min",
...
)

fit(monitor="val_acc", mode="max") (or the same via TrainerConfig.monitor/mode) is honored for stopping, but the best-checkpoint selection stays hardcoded to val_loss/min — and after training the model's weights are restored from that checkpoint (fit.py:525-529). So a user who monitors accuracy silently gets the best-val-loss weights, not the best-accuracy weights. Neither docstring (monitor : The metric to monitor for early stopping and checkpointing in the estimator bases) mentions the divergence. The same pattern exists in deeptab/models/lss_base.py:308-318.

To Reproduce

importnumpyasnp, pandasaspdfromdeeptab.modelsimportMLPClassifierX=pd.DataFrame({"a": np.random.randn(200)})
y= (X["a"] >0).astype(int).valuesm=MLPClassifier()
m.fit(X, y, max_epochs=10, monitor="val_acc", mode="max",
val_metrics={"val_acc": "accuracy"})
# EarlyStopping tracks val_acc/max; ModelCheckpoint still tracks val_loss/min,# and the restored weights are the best-val_loss epoch, not the best-val_acc epoch.

Expected behavior
ModelCheckpoint should use the same monitor/mode as EarlyStopping (they come from the same user parameter), or the docs should state explicitly that weight restoration is always by validation loss.

Screenshots
n/a

Desktop (please complete the following information):

  • OS: macOS (Darwin 25.5.0, arm64)
  • Python version: 3.11.15
  • deeptab Version: 2.0.0 (main @ 4e6a359)

Additional context
Follow-up from the same review as #409#424; complements #411 (checkpoint dirpath/collision issues) — this one is about which checkpoint is selected, that one about where it is written.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions