Describe the bug
EarlyStopping and ModelCheckpoint monitor different metrics, and the final weights come from the checkpoint (deeptab/models/_mixins/fit.py:481-494):
early_stop_callback=EarlyStopping(
monitor=monitor, min_delta=0.00, patience=patience, verbose=False, mode=mode
)
checkpoint_callback=ModelCheckpoint(
monitor="val_loss",
mode="min",
...
)
fit(monitor="val_acc", mode="max") (or the same via TrainerConfig.monitor/mode) is honored for stopping, but the best-checkpoint selection stays hardcoded to val_loss/min — and after training the model's weights are restored from that checkpoint (fit.py:525-529). So a user who monitors accuracy silently gets the best-val-loss weights, not the best-accuracy weights. Neither docstring (monitor : The metric to monitor for early stopping and checkpointing in the estimator bases) mentions the divergence. The same pattern exists in deeptab/models/lss_base.py:308-318.
To Reproduce
importnumpyasnp, pandasaspdfromdeeptab.modelsimportMLPClassifierX=pd.DataFrame({"a": np.random.randn(200)})
y= (X["a"] >0).astype(int).valuesm=MLPClassifier()
m.fit(X, y, max_epochs=10, monitor="val_acc", mode="max",
val_metrics={"val_acc": "accuracy"})
# EarlyStopping tracks val_acc/max; ModelCheckpoint still tracks val_loss/min,# and the restored weights are the best-val_loss epoch, not the best-val_acc epoch.Expected behavior
ModelCheckpoint should use the same monitor/mode as EarlyStopping (they come from the same user parameter), or the docs should state explicitly that weight restoration is always by validation loss.
Screenshots
n/a
Desktop (please complete the following information):
- OS: macOS (Darwin 25.5.0, arm64)
- Python version: 3.11.15
- deeptab Version: 2.0.0 (main @ 4e6a359)
Additional context
Follow-up from the same review as #409–#424; complements #411 (checkpoint dirpath/collision issues) — this one is about which checkpoint is selected, that one about where it is written.
Describe the bug
EarlyStoppingandModelCheckpointmonitor different metrics, and the final weights come from the checkpoint (deeptab/models/_mixins/fit.py:481-494):fit(monitor="val_acc", mode="max")(or the same viaTrainerConfig.monitor/mode) is honored for stopping, but the best-checkpoint selection stays hardcoded toval_loss/min— and after training the model's weights are restored from that checkpoint (fit.py:525-529). So a user who monitors accuracy silently gets the best-val-loss weights, not the best-accuracy weights. Neither docstring (monitor : The metric to monitor for early stopping and checkpointingin the estimator bases) mentions the divergence. The same pattern exists indeeptab/models/lss_base.py:308-318.To Reproduce
Expected behavior
ModelCheckpointshould use the samemonitor/modeasEarlyStopping(they come from the same user parameter), or the docs should state explicitly that weight restoration is always by validation loss.Screenshots
n/a
Desktop (please complete the following information):
Additional context
Follow-up from the same review as #409–#424; complements #411 (checkpoint dirpath/collision issues) — this one is about which checkpoint is selected, that one about where it is written.