Fix: numpy scalars break JSON export of results - #3
Open
alexisdubs wants to merge 1 commit into
Open
alexisdubs wants to merge 1 commit into
alexisdubs wants to merge 1 commit into
Conversation
The sanitizer before json.dump converts np.ndarray values but not numpy scalar types, so any model storing an integer hyperparameter as np.int64 raised "TypeError: Object of type int64 is not JSON serializable". PLS and SPLS both store n_components this way. The model trains fine and the pickle is written first, so results are not lost, but the run ends in a traceback and no JSON is produced. This reproduces on any environment with NumPy 2.x. Passing default= to json.dump converts numpy scalars via .item(). It only affects objects the encoder would otherwise reject, so output for currently-working models is unchanged (verified: OLS and LCEN test R^2 identical before and after). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
main_SPA()sanitizesfitting_resultbefore writing the JSON results file, but the loop atCode-SPA/SPA.py:565-570only convertsnp.ndarrayvalues. Numpy scalars pass through untouched, so any model that stores an integer hyperparameter asnp.int64crashes when the results are saved.PLSandSPLSboth storen_componentsthis way.The model itself trains correctly, and the pickle is written before the JSON, so results are not lost — but the run ends in a traceback and no
.jsonfile is produced.Reproduces on NumPy 2.x. Seen on Python 3.11.9 / NumPy 2.4.6 / pandas 3.0.5 / scikit-learn 1.9.0, Windows:
Fix
Pass
default=tojson.dumpso numpy scalars are converted via.item().np.genericis the base class for every numpy scalar type, so this covers int64, float64, bool_ and the rest in one place rather than enumerating them.The handler only ever sees objects the encoder was already going to reject, so output for models that currently serialize successfully is byte-for-byte unchanged. Verified by running
Examples/Concrete_data.csv(80/20 split) before and after the change:🤖 Generated with Claude Code