bark.cpp

Roadmap / encodec.cpp / ggml

Inference of SunoAI's bark model in pure C/C++.

Description

With bark.cpp, our goal is to bring real-time realistic multilingual text-to-speech generation to the community.

• Plain C/C++ implementation without dependencies
• AVX, AVX2 and AVX512 for x86 architectures
• CPU and GPU compatible backends
• Mixed F16 / F32 precision
• 4-bit, 5-bit and 8-bit integer quantization
• Metal and CUDA backends

Models supported

• Bark Small
• Bark Large

Models we want to implement! Please open a PR :)

• AudioCraft (#62)
• AudioLDM2 (#82)
• Piper (#135)

Demo on Google Colab (#95)

---

Here is a typical run using bark.cpp:

./main -p"This is an audio generated by bark.cpp"____
/ /___________/ /_______________
/ __ \/ __ `/ ___/ //_/ / ___/ __ \/ __ \
/ /_/ / /_/ / / / ,< _ / /__/ /_/ / /_/ /
/_.___/\__,_/_/ /_/|_| (_) \___/ .___/ .___/
/_/ /_/
bark_tokenize_input: prompt: 'This is an audio generated by bark.cpp'bark_tokenize_input: numberoftokensinprompt = 513, first8tokens: 2079520172201993373358966202032816920222Generatingsemantictokens: 17%
bark_print_statistics: sampletime = 10.98ms / 138tokensbark_print_statistics: predicttime = 614.96ms / 4.46mspertokenbark_print_statistics: totaltime = 633.54msGeneratingcoarsetokens: 100%
bark_print_statistics: sampletime = 3.75ms / 410tokensbark_print_statistics: predicttime = 3263.17ms / 7.96mspertokenbark_print_statistics: totaltime = 3274.00msGeneratingfinetokens: 100%
bark_print_statistics: sampletime = 38.82ms / 6144tokensbark_print_statistics: predicttime = 4729.86ms / 0.77mspertokenbark_print_statistics: totaltime = 4772.92mswrite_wav_on_disk: Numberofframeswritten = 65600.main: loadtime = 324.14msmain: evaltime = 8806.57msmain: totaltime = 9131.68ms

Here is a video of Bark running on the iPhone:

ouput.mp4

Usage

Here are the steps to use Bark.cpp

Get the code

git clone --recursive https://github.com/PABannier/bark.cpp.git
cd bark.cpp
git submodule update --init --recursive

Build

In order to build bark.cpp you must use CMake:

mkdir build
cd build
# To enable nvidia gpu, use the following option# cmake -DGGML_CUBLAS=ON ..
cmake ..
cmake --build . --config Release

Prepare data & Run

# Install Python dependencies
python3 -m pip install -r requirements.txt
# Download the Bark checkpoints and vocabulary
python3 download_weights.py --out-dir ./models --models bark-small bark
# Convert the model to ggml format
python3 convert.py --dir-model ./models/bark-small --use-f16
# run the inference
./build/examples/main/main -m ./models/bark-small/ggml_weights.bin -p "this is an audio generated by bark.cpp" -t 4

(Optional) Quantize weights

Weights can be quantized using the following strategy: q4_0, q4_1, q5_0, q5_1, q8_0.

Note that to preserve audio quality, we do not quantize the codec model. The bulk of the computation is in the forward pass of the GPT models.

./build/examples/quantize/quantize ./ggml_weights.bin ./ggml_weights_q4.bin q4_0

Seminal papers

• Bark
  • Text Prompted Generative Audio
• Encodec
  • High Fidelity Neural Audio Compression
• GPT-3
  • Language Models are Few-Shot Learners

Contributing

bark.cpp is a continuous endeavour that relies on the community efforts to last and evolve. Your contribution is welcome and highly valuable. It can be

• bug report: you may encounter a bug while using bark.cpp. Don't hesitate to report it on the issue section.
• feature request: you want to add a new model or support a new platform. You can use the issue section to make suggestions.
• pull request: you may have fixed a bug, added a features, or even fixed a small typo in the documentation, ... you can submit a pull request and a reviewer will reach out to you.

Coding guidelines

• Avoid adding third-party dependencies, extra files, extra headers, etc.
• Always consider cross-compatibility with other operating systems and architectures