Describe the bug
GammaDeviance has the sign of its log term inverted (deeptab/metrics/distributional.py:341):
returnfloat(2.0*np.mean(np.log(y_true/mu) + (y_true-mu) /mu))
The gamma deviance is 2·mean(log(µ/y) + y/µ − 1); the code computes 2·mean(log(y/µ) + y/µ − 1). Because both forms are 0 at µ = y, the existing unit test (tests/test_metrics.py:367) passes — but for any imperfect prediction the metric disagrees with sklearn.metrics.mean_gamma_deviance (0.341 vs 0.135 on a small example), and worse, it is unbounded below: predicting µ = 1000·y scores −14.6, so the metric rewards unbounded over-prediction.
GammaDeviance is the primary metric for lss:gamma in METRIC_REGISTRY, so any model selection or HPO minimizing it is driven toward degenerate over-predicting models.
To Reproduce
importnumpyasnpfromsklearn.metricsimportmean_gamma_deviancefromdeeptab.metrics.distributionalimportGammaDeviancey=np.array([1.0, 2.0, 3.0])
mu=np.array([2.0, 2.0, 2.0])
print(GammaDeviance()(y, mu[:, None])) # 0.341...print(mean_gamma_deviance(y, mu)) # 0.135...mu=np.full(3, 1000.0)
print(GammaDeviance()(y, mu[:, None])) # ~ -14.6 (deviance must be >= 0)
Expected behavior
2·mean(log(µ/y) + y/µ − 1) — non-negative, zero iff µ = y, matching sklearn.
Screenshots
n/a
Desktop (please complete the following information):
- OS: macOS (Darwin 25.5.0, arm64)
- Python version: 3.11.15
- deeptab Version: 2.0.0 (main @ 4e6a359)
Additional context
Note the test-design lesson: a deviance test that only checks the perfect-prediction case cannot detect a sign error, since both signs give 0 there. Comparing against sklearn on imperfect predictions catches it. (Separately, whether col 0 of the gamma LSS head is even the mean is part of the LSS metrics-wiring umbrella issue — the gamma head outputs [shape, rate].)
Describe the bug
GammaDeviancehas the sign of its log term inverted (deeptab/metrics/distributional.py:341):The gamma deviance is
2·mean(log(µ/y) + y/µ − 1); the code computes2·mean(log(y/µ) + y/µ − 1). Because both forms are 0 atµ = y, the existing unit test (tests/test_metrics.py:367) passes — but for any imperfect prediction the metric disagrees withsklearn.metrics.mean_gamma_deviance(0.341 vs 0.135 on a small example), and worse, it is unbounded below: predictingµ = 1000·yscores −14.6, so the metric rewards unbounded over-prediction.GammaDevianceis the primary metric forlss:gammainMETRIC_REGISTRY, so any model selection or HPO minimizing it is driven toward degenerate over-predicting models.To Reproduce
Expected behavior
2·mean(log(µ/y) + y/µ − 1)— non-negative, zero iff µ = y, matching sklearn.Screenshots
n/a
Desktop (please complete the following information):
Additional context
Note the test-design lesson: a deviance test that only checks the perfect-prediction case cannot detect a sign error, since both signs give 0 there. Comparing against sklearn on imperfect predictions catches it. (Separately, whether col 0 of the gamma LSS head is even the mean is part of the LSS metrics-wiring umbrella issue — the gamma head outputs
[shape, rate].)