audiogen.cpp

High-performance C++17 GGML inference engine for Dasheng-AudioGen using pre-converted and quantized GGUF weights audiohacking/dasheng-audiogen-gguf

⚠️ Experimental - use at your own risk!

About AudioGen

Dasheng-AudioGen is a unified audio generation model that can jointly synthesize intelligible speech, music, sound effects, and environmental acoustics from text descriptions.

dasheng-audiogen-demo-video.mp4

Quick Start

# Clone and build
git clone https://github.com/audiohacking/audiogen.cpp
cd audiogen.cpp
git submodule update --init --recursive
make metal # or: make cpu# Download models (~6GB)
make download-models
# Generate audio
./build-metal/dasheng-audiogen \
models/t5_encoder.gguf models/dit.gguf models/vocoder.gguf models/spiece.model \
--caption "A dog barking loudly" --output dog.wav

Performance

With 25 steps and 10s generations C++ is ~3.7x faster than Python (apple metal)

Prompt	Python	C++	Speedup
<|caption|> A dog barking loudly	22.67s	5.97s	3.80x
<|caption|> Birds chirping in a forest <	22.22s	6.06s	3.66x
<|caption|> Thunder and rain <|music|> d	22.35s	6.07s	3.68x
TOTAL	67.24s	18.11s	3.71x

Per-step average: Python=897ms, C++=241ms

Build

make metal # Metal backend (macOS)
make cpu # CPU-only build
make cuda # CUDA backend (Linux)

Download Models

# F16 models (~5.7GB) - best quality
make download-models
# Q8 models (~3.8GB) - good quality, 33% smaller
make download-models-q8
# Q4 models (~2.8GB) - smallest, 51% smaller
make download-models-q4

Or manually:

hf download audiohacking/dasheng-audiogen-gguf --local-dir models/

Usage

Basic

./build-metal/dasheng-audiogen \
models/t5_encoder.gguf models/dit.gguf models/vocoder.gguf models/spiece.model \
--caption "Rain falling on a window" --output rain.wav

Prompt Composition

Combine tags for richer generation:

./build-metal/dasheng-audiogen \
models/t5_encoder.gguf models/dit.gguf models/vocoder.gguf models/spiece.model \
--caption "A peaceful morning" \
--music "soft piano melody" \
--env "birds chirping" \
--output morning.wav

Tags: --caption, --speech, --asr, --sfx, --music, --env

Options

Option	Default	Description
--steps N	25	Diffusion steps
--duration SECS	10	Audio duration in seconds
--cfg SCALE	3.0	Guidance scale
--seed N	random	Random seed
--threads N	4	CPU threads

Batch Processing

cat > prompts.txt << 'EOF'<|caption|> A dog barking<|caption|> Thunder rollingEOF
./build-metal/dasheng-audiogen \
models/t5_encoder.gguf models/dit.gguf models/vocoder.gguf models/spiece.model \
--batch prompts.txt --output-dir output/

Converting Models

To convert from original weights instead of downloading:

pip install -r convert/requirements.txt
make convert

License

Apache-2.0