Vocabulary Classifier

Supervised learning for text classification based on vocabulary analysis for NodeJS. It depends on redis and is supposed to handle huge amounts of data for training.

API

newVocabularyClassifier(redisClient,ngrams).normalizeText(text).trainLabel(labelName,text,callback).classifyText(labelsArray,text,callback).removeLabel(labelName,callback).getLabelWordCount(labelName,callback).getLabels(callback)

Usage

//The classifier depends on redisconstredis=require("redis");//Import the classifierconstVocabularyClassifier=require("./VocabularyClassifier.js");//Create a redis clientconstclient=redis.createClient();//Create a instance of the Classifierletclassifier=newVocabularyClassifier(client,1);//Wait until redis is connectedclient.on('connect',function(){//Train the classifier with labeled dataclassifier.trainLabel("german","dies ist ein deutscher text text",function(){//Train it again with different labeld dataclassifier.trainLabel("english","this is an english text",function(){//Classify a new textclassifier.classifyText(["german","english"],"dies ist",function(wordForWord,reduced){//Output the result for every wordconsole.log(JSON.stringify(wordForWord,null,3));//Output the result for the entire textconsole.log(reduced);// > [ { label: 'german', score: 0.875 },// { label: 'english', score: 0.125 } ]//Output the trained labelsclassifier.getLabels(function(result){console.log(result);// > [ 'english', 'german' ]//Remove the labels to free the redis databaseclassifier.removeLabel("german");classifier.removeLabel("english");});});});});});

Example result of .classifyText(...)

The word for word result

[{"word": "text","result": [{"label": "german","partOfLanguage": 0.3333333333333333,"score": 0.625},{"label": "english","partOfLanguage": 0.2,"score": 0.375}]},
...
]

The overall text result. This text is obviously german.

[{label: 'german',score: 0.875},{label: 'english',score: 0.125}]

Dependencies

• redis