Skip to content

Fix: numpy scalars break JSON export of results - #3

Open
alexisdubs wants to merge 1 commit into
PedroSeber:masterfrom
alexisdubs:fix-numpy-json
Open

alexisdubs wants to merge 1 commit into
PedroSeber:masterfrom
alexisdubs:fix-numpy-json

Conversation

@alexisdubs

@alexisdubs alexisdubs commented Sep 4, 2026

Copy link
Copy Markdown

main_SPA() sanitizes fitting_result before writing the JSON results file, but the loop at Code-SPA/SPA.py:565-570 only converts np.ndarray values. Numpy scalars pass through untouched, so any model that stores an integer hyperparameter as np.int64 crashes when the results are saved. PLS and SPLS both store n_components this way.

Traceback (most recent call last):
  File "SPA.py", line 581, in main_SPA
    json.dump(fr2, f, indent = 4)
  ...
TypeError: Object of type int64 is not JSON serializable

The model itself trains correctly, and the pickle is written before the JSON, so results are not lost — but the run ends in a traceback and no .json file is produced.

Reproduces on NumPy 2.x. Seen on Python 3.11.9 / NumPy 2.4.6 / pandas 3.0.5 / scikit-learn 1.9.0, Windows:

import SPA
SPA.main_SPA('train.csv', test_data='test.csv', model_name=['PLS'], cv_method='KFold', K_fold=3)

Fix

Pass default= to json.dump so numpy scalars are converted via .item(). np.generic is the base class for every numpy scalar type, so this covers int64, float64, bool_ and the rest in one place rather than enumerating them.

The handler only ever sees objects the encoder was already going to reject, so output for models that currently serialize successfully is byte-for-byte unchanged. Verified by running Examples/Concrete_data.csv (80/20 split) before and after the change:

Model test R² before test R² after
OLS 0.5951 0.5951
LCEN 0.8841 0.8841
PLS crash 0.5934
SPLS crash 0.5934

🤖 Generated with Claude Code

The sanitizer before json.dump converts np.ndarray values but not numpy
scalar types, so any model storing an integer hyperparameter as np.int64
raised "TypeError: Object of type int64 is not JSON serializable".
PLS and SPLS both store n_components this way.

The model trains fine and the pickle is written first, so results are not
lost, but the run ends in a traceback and no JSON is produced. This
reproduces on any environment with NumPy 2.x.

Passing default= to json.dump converts numpy scalars via .item(). It only
affects objects the encoder would otherwise reject, so output for
currently-working models is unchanged (verified: OLS and LCEN test R^2
identical before and after).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant