Monotone feature constraints in the distilled whitebox - #21
Merged
Conversation
) Training: train_whitebox(monotone_constraints=...) switches the whitebox backend to HistGradientBoostingRegressor (monotonic_cst enforced during growth); the unconstrained GradientBoostingRegressor path is untouched. New sklearn_hist extractor with random-row parity validation; categorical splits are refused at the model level. Verification is trainer-independent: build_artifact re-checks the declared directions against the quantized integer trees (per-tree, any depth) and refuses to build on violation, then records the signs as model.monotone_constraints — optional, hash-covered, spec section 3.1. Validation gains check 9, re-verifying from the artifact alone. At depth <= 2, scorecard_monotone_report certifies the aggregate direction on the printed scorecard tables via worst-case bin-to-bin increments. sweep_whitebox(monotone_constraints=...) measures the monotonicity premium. Docs: tuning guide section, FAQ entry, spec 3.1, nine-check tables. Tests use a teacher with a deliberate local inversion so the unconstrained whitebox provably violates and the verifier provably catches it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes#8
What this adds
Training —
train_whitebox(monotone_constraints=...)accepts a per-feature sequence of-1/0/+1, an index-keyed dict, or a name-keyed dict whenXis a DataFrame. Any nonzero sign switches the backend toHistGradientBoostingRegressor(the only sklearn GBM that enforcesmonotonic_cstduring tree growth). Without constraints the classicGradientBoostingRegressorpath is untouched — the committed reference artifact hash is unchanged.Extraction — new
sklearn_histfamily walker with the same random-row parity gate as the other families (measured max error vsmodel.predict: 0.0 on 200 rows). HGB leaves arrive pre-shrunk (learning_rate=1.0in the artifact); categorical splits are refused at the model level because they remapfeature_idxinside the predictor nodes.Verification, independent of the trainer — the design center of the PR.
build_artifact(monotone_constraints=...)re-verifies the declaration against the quantized integer trees (per-tree, any depth — a sum of monotone trees is monotone) and refuses to build on any violation, whatever produced the model. Verified signs are recorded asmodel.monotone_constraints: optional, hash-covered, spec §3.1,schema_versionunchanged.Enforcement ladder — train (enforced) → build (verify-or-raise) → validation check 9 (re-verifies from the artifact alone, no data needed) → at depth ≤ 2,
scorecard_monotone_reportcertifies the aggregate direction on the printed scorecard tables via worst-case bin-to-bin increments — a check a validator can repeat in a spreadsheet.Measurement —
sweep_whitebox(monotone_constraints=...)for the monotonicity premium; the tuning guide shows the two-sweep diff.Tests (17 new)
The fixture teacher has a deliberate local inversion (
x0 − 1.2·sin(2.5·x0)dips while trending up), so the tests have teeth: the unconstrained whitebox provably violates and the verifier provably catches it; the constrained backend provably doesn't. Also covered: extraction parity, build refusal, hash coverage of the recorded field, constrained rebuild determinism (hash-identical), check 9 pass/skip/fail (fail via a hand-tampered declaration), scorecard certification both ways, categorical guard, and constraint-form normalization.test_validate's fixture now declares the (genuinely monotone) teacher's directions so its all-nine-checks-run assertion holds.Notes