javascript

whisper.cpp

Node.js package for Whisper speech recognition

Package: https://www.npmjs.com/package/whisper.cpp

Details

The performance is comparable to when running whisper.cpp in the browser via WASM.

The API is currently very rudimentary: bindings/javascript/emscripten.cpp

For sample usage check tests/test-whisper.js

Package building + test

# load emscriptensource /path/to/emsdk/emsdk_env.sh
# clone repo
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
# grab base.en model
./models/download-ggml-model.sh base.en
# prepare PCM sample for testing
ffmpeg -i samples/jfk.wav -f f32le -acodec pcm_f32le samples/jfk.pcmf32
# build
mkdir build-em &&cd build-em
emcmake cmake .. && make -j
# run test
node --experimental-wasm-threads --experimental-wasm-simd ../tests/test-whisper.js
# publish npm package
make publish-npm

Sample run

$node --experimental-wasm-threads --experimental-wasm-simd ../tests/test-whisper.jswhisper_model_load: loadingmodelfrom'whisper.bin'whisper_model_load: n_vocab = 51864whisper_model_load: n_audio_ctx = 1500whisper_model_load: n_audio_state = 512whisper_model_load: n_audio_head = 8whisper_model_load: n_audio_layer = 6whisper_model_load: n_text_ctx = 448whisper_model_load: n_text_state = 512whisper_model_load: n_text_head = 8whisper_model_load: n_text_layer = 6whisper_model_load: n_mels = 80whisper_model_load: f16 = 1whisper_model_load: type = 2whisper_model_load: adding1607extratokenswhisper_model_load: mem_required = 506.00MBwhisper_model_load: ggmlctxsize = 140.60MBwhisper_model_load: memorysize = 22.83MBwhisper_model_load: modelsize = 140.54MBsystem_info: n_threads = 8 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | NEON = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 1 | BLAS = 0 | operator(): processing176000samples, 11.0sec, 8threads, 1processors, lang = en, task = transcribe ...
[00:00:00.000 --> 00:00:11.000] AndsomyfellowAmericans, asknotwhatyourcountrycandoforyou, askwhatyoucandoforyourcountry.
whisper_print_timings: loadtime = 162.37mswhisper_print_timings: meltime = 183.70mswhisper_print_timings: sampletime = 4.27mswhisper_print_timings: encodetime = 8582.63ms / 1430.44msperlayerwhisper_print_timings: decodetime = 436.16ms / 72.69msperlayerwhisper_print_timings: totaltime = 9370.90ms