Self-incremental learning vector quantization

This implementation is based on the below paper. Self-incremental learning vector quantization with human cognitive biases (https://www.nature.com/articles/s41598-021-83182-4).

Compatible with Python3.6 and above.

Requirements

• NumPy
• scikit-learn
• matplotlib

Quickstart

This is the minimum code to run SilvqModel.

importnumpyasnpfromsklearn.metricsimportaccuracy_scorefromsklearn.model_selectionimporttrain_test_splitfromsklearn.datasetsimportload_irisfromlvqimportSilvqModel# Load datasetiris=load_iris()
x=iris.datay=iris.target# Split dataset into training set and test setx_train, x_test, y_train, y_test=train_test_split(x, y, test_size=0.2, random_state=8, shuffle=True, stratify=y)
# Generating modelmodel=SilvqModel(x.shape[1], theta=0.8, bias_type='ls')
# Training the modelmodel.fit(x_train, y_train, epochs=1)
# Predict the response for test datasety_predict=model.predict(x_test)
# Evaluating the modelprint('Accuracy: %.3f'%accuracy_score(y_test, y_predict))

For more information, score_iris_silvq.py.

Techniques

The following techniques can be used to improve accuracy.

fromlvqimportSilvqModelfromlvq.utilsimportchoice_prototypesinitial_prototypes=choice_prototypes(x_train, y_train, prototypes_per_class=1, random_state=None)
model=SilvqModel(x.shape[1], theta=0.5, bias_type='ls', initial_prototypes=initial_prototypes)
model.fit(x_train, y_train, epochs=30)

For more information, score_wine_silvq.py.

Conformal prediction

Conformal prediction can be performed.

importnumpyasnpfromsklearn.metricsimportaccuracy_scorefromsklearn.model_selectionimporttrain_test_splitfromsklearn.datasetsimportload_digitsfromlvqimportSilvqModelfromlvq.utilsimportconformal_predict, accuracy_score_conformal_predictions# Load datasetdigits=load_digits()
x=digits.datay=digits.target# Split dataset into training set and test setx_train, x_test, y_train, y_test=train_test_split(x, y, test_size=0.2, random_state=8, shuffle=True, stratify=y)
# Split training dataset into training set and calibration setx_train, x_calib, y_train, y_calib=train_test_split(x_train, y_train, test_size=0.2, random_state=8, shuffle=True, stratify=y_train)
# Generating modelmodel=SilvqModel(x.shape[1], theta=0.5, bias_type='ls')
# Training the modelmodel.fit(x_train, y_train, epochs=30)
# Predict the response for test datasety_predict=model.predict(x_test)
# Evaluate the model's accuracyprint('Accuracy: %.3f'%accuracy_score(y_test, y_predict))
# Conformal predictionconformal_predictions=conformal_predict(model, x_calib, y_calib, x_test, confidence_level=0.99)
# Evaluate the model's accuracy in conformal predictionsprint('Conformal prediction accuracy: %.3f'%accuracy_score_conformal_predictions(y_test, conformal_predictions))
# Display the results of 10 conformal predictionsprint('** Displaying 10 sample conformal predictions')
foridxinrange(10):
print('Test{}: True Label: {}, Predicted: {}'.format(idx, y_test[idx], conformal_predictions[idx]))
print('*********************************************')

For more information, score_conformal_prediction_digits_silvq.py.

Data compression & noise reduction

You can use this model to perform data compression and noise reduction.

importnumpyasnpfromsklearn.metricsimportaccuracy_scorefromsklearn.model_selectionimporttrain_test_splitfromsklearn.datasetsimportload_breast_cancerfromlvqimportSilvqModel# Load datasetbreast_cancer=load_breast_cancer()
x=breast_cancer.datay=breast_cancer.target# Split dataset into training set and test setx_train, x_test, y_train, y_test=train_test_split(x, y, test_size=0.2, random_state=8, shuffle=True, stratify=y)
# Generating modelmodel=SilvqModel(x.shape[1], theta=0.5, bias_type='ls')
# Training the modelmodel.fit(x_train, y_train, epochs=30)
# Predict the response for test datasety_predict=model.predict(x_test)
# Evaluating the modelprint('** Original ****************************')
print('Accuracy: %.3f'%accuracy_score(y_test, y_predict))
print('Number of prototypes: {}'.format(model.n_prototypes))
# Noise reductionmodel.delete_prototype(140)
# Predict the response for test datasety_predict=model.predict(x_test)
# Evaluating the modelprint('** Noise reduction *********************')
print('Accuracy: %.3f'%accuracy_score(y_test, y_predict))
print('Number of prototypes: {}'.format(model.n_prototypes))

For more information, export_compression_artificial_dataset2_silvq.py, export_compression_and_noise_reduction_artificial_dataset2_silvq.py, and score_noise_reduction_breast_cancer_silvq.py.

Plot prototypes

You can use plot2d to visualize the prototype of SilvqModel.

fromlvq.utilsimportplot2dplot2d(model, x, y)

The plot shows the target class of each data point (big circle) and which class was predicted (smaller circle). It also shows the prototypes (diamond).

For more information, plot2d_artificial_dataset1_silvq.py.

For more information, plot2d_artificial_dataset2_silvq.py.

For more information, plot2d_moons_silvq.py.

Note

Self-incremental learning vector quantization (SILVQ) provides a learning algorithm that can be intuitively understood. To improve SILVQ performance, distances other than the Euclidean distance, such as cosine distance, may be used instead. As with advanced LVQ algorithms designing models to strictly minimize classification errors may be possible. However, human learning is variable and compound; it is not exclusively based on minimizing the classification errors.

This research aims to contribute to both the computer science and cognitive science fields and hopes to support the research of explainable artificial intelligence (XAI) to address the black box problem of machine learning.

License

This is free and open-source software licensed under the 3-clause BSD license.