Licensed under the Apache License 2.0 psychcoherence@gmail.com OR therealmichaelmaillet@gmail.com
SSGS (Spectral-State Guided Synthesis) is a hybrid generative audio model that combines linear predictive coding (LPC), hidden Markov models (HMMs), spectral clustering, global graph search, and a physically-inspired Karplus–Strong excitation engine into a single synthesis framework.
Unlike classical systems that commit to a single paradigm (neural, statistical,
or physical), SSGS deliberately smashes algorithms together and extracts the
useful computational primitives from each.
The result is a model that:
- learns spectral structure from real signals,
- organizes that structure into Markov states,
- decodes the optimal state trajectory over time via A* search,
- generates excitation signals using string-model dynamics, and
- reconstructs audio by filtering excitation through LPC envelopes.
SSGS is not meant to imitate any existing model family.
It is a new hybrid class: detailed like DSP, global like HMMs, and expressive like
physical models.
SSGS is built using the same philosophy behind the Symbo project:
Break apart many algorithms, strip them to their mathematical essence, recombine only the primitives that are actually useful.
Instead of inheriting algorithms whole (e.g., a standard HMM, standard LPC vocoder, standard Karplus–Strong), SSGS takes:
- LPC → spectral envelope representation
- HMM → temporal clustering + statistical state transitions
- EM → unsupervised parameter refinement
- A* search → globally optimal state sequence
- Karplus–Strong → natural resonance and excitation noise
- Heuristics → spectral smoothness constraints
- Graph theory → prune invalid or degenerate state structures
SSGS then recomposes these pieces into a single generative pipeline that didn’t exist before.
This approach is extremely flexible: swap the envelope model, swap the excitation, modify the search heuristic — the system keeps working.
The training signal is segmented and analyzed with LPC to extract:
- LPC coefficients
- excitation residual
- power envelope
These become the feature vectors for clustering.
Frames are embedded into a feature space (typically derived from LPC spectra). States are initialized using k-means, then upgraded into a full HMM with:
- initial state distribution
- transition matrix
- mean vectors and covariance matrices for each state
A custom EM implementation updates:
- state responsibilities (γ)
- pairwise transitions (ξ)
- transition probabilities
- Gaussian parameters (means, covariances)
Result: the HMM becomes a structured map of repeating spectral “modes.”
To prevent degenerate solutions, SSGS analyzes the transition graph:
- identifies strongly connected components (SCCs)
- removes invalid or isolated states
- ensures state sequences remain musically plausible
Instead of using Viterbi (which is greedy and purely local), SSGS uses A*:
- cost function = negative log-likelihood + spectral smoothness heuristic
- ensures global consistency in the decoded state path
- supports long-range structure better than classical decoding
The decoded state sequence modulates a Karplus–Strong string model, producing a dynamic excitation that is:
- rich in overtones
- noisy where appropriate
- resonant and evolving
Finally, excitation is passed through the LPC filters of each decoded state:
- reconstructs spectral envelopes
- restores formants and resonances
- yields new but structurally coherent audio
This is how SSGS generates novel signals even from short or simple training data.
After training you can persist the learned parameters in a compact archive:
ssgs.export_model("models/ssgs_baseline", use_compression=True, pack_covariances=True).npzoutput uses ZIP compression by default whenuse_compression=True.- Covariance matrices are serialized as lower-triangular slices to halve their stored size.
- Set
include_training_artifacts=Trueif you also need cached LPC and residual buffers for later analysis. - Reload the model with
SpectralStateGuidedSynthesis.load_model(path).
For long-running training, SSGS also supports .safetensors checkpoints:
ssgs.save_checkpoint("models/checkpoint_epoch_5.safetensors")
ssgs=SpectralStateGuidedSynthesis.load_checkpoint("models/checkpoint_epoch_5.safetensors")Checkpoints can optionally persist adaptive statistics so continuous learning can resume without reprocessing previous audio.
SSGS now offers enhanced generative capabilities through configurable state counts:
- Standard (16 states): Original baseline configuration
- Enhanced (34 states): 112% increase - recommended for most applications
- Maximum (42 states): 163% increase - highest generative capacity
# Enhanced generative capabilities (default)ssgs=SpectralStateGuidedSynthesis(n_states=34)
# Maximum generative capabilitiesssgs=SpectralStateGuidedSynthesis(n_states=42)
# Original baselinessgs=SpectralStateGuidedSynthesis(n_states=16)More states = more spectral patterns the model can learn and generate = richer, more diverse output.
SSGS produces time-domain and spectral visualizations like this:
For a deeper comparison diagnostic, see ssgs_analysis.png.
The generated signal is not a copy —
it is a new trajectory through learned spectral states.
Install dependencies:
pip install -r requirements.txtRun the demo training + generation script:
python test_ssgs.pyRun the enhanced capabilities demo (defaults to 34 states):
python demo_enhanced_capabilities.pyTrain on folders of audio with checkpointing:
python train_on_folder.py --folders training_001 training_002SSGS can adapt to new audio using persistent statistics:
ssgs.adapt_to_audio(new_audio, sample_rate=16000, adaptation_rate=0.5, memory_blend=0.3)Adaptation can be exported through .npz or .safetensors artifacts and reloaded later.
Use built-in nearest-neighbor search over frame features for exploration and analysis:
index=ssgs.build_feature_index(feature_space="lpc", normalize=True)
result=ssgs.query_similar_frames(frame_idx=42, k=8, feature_space="lpc")
vector_result=ssgs.search_by_feature_vector(
vector=index.tree.data[0],
k=5,
feature_space="lpc",
)The returned FrameQueryResult includes indices, optional distances, and frame metadata.
The numeric hot loops are also implemented as a small, dependency-light C
library (ssgs_native.c) and exposed through ssgs_native.py. ssgs.py
automatically uses them when libssgs_native.so is loadable, and falls back to
the existing SciPy/NumPy paths otherwise — so the package still works on a
machine with no compiler installed.
Kernels covered: LPC (Levinson–Durbin), Gaussian emission (hand Cholesky + forward solve), EM forward–backward, and the smoothness-first Viterbi decode. The decode is the dominant synthesis cost (a pure-Python triple loop in the SciPy build); the compiled kernel is ~170× faster there, with identical output:
| kernel | SciPy/NumPy | C | speedup |
|---|---|---|---|
| LPC (Levinson–Durbin) | 0.172 s | 0.172 s | 1.0× |
| Gaussian emission | 0.0014 s | 0.0012 s | 1.2× |
| decode (2000 frames) | 0.4998 s | 0.0029 s | 170× |
The C kernels are numerically identical to ssgs.py (enforced by
test_regression_guard.py). Build the library with:
make # produces libssgs_native.so (requires gcc)A pure-NumPy reference implementation lives in ssgs_core.py and is used by the
regression guard to prove equivalence even when the compiled library is absent.
Run individual checks as standalone scripts:
python test_ssgs.py
python test_checkpoints.py
python test_adaptive_persistence.py
python test_data_indexing.py
python test_gain_control.py
python test_generative_capabilities.py
python test_utils.pyNumeric-equivalence and performance gates:
python ssgs_core.py # pure-NumPy core self-test (equivalence oracle)
python ssgs_native.py # compiled-kernel verify + benchmark
python test_regression_guard.py # native load + core/pipeline equivalencenumpy>=1.21.0
scipy>=1.7.0
matplotlib>=3.4.0
soundfile>=0.10.0
PyWavelets>=1.1.0
safetensors>=0.4.0
Optional: gcc and make to build the native backend (libssgs_native.so).
If unavailable, SSGS falls back to the SciPy/NumPy paths automatically.
ssgs.py # Full model implementation
ssgs_core.py # Pure-NumPy reference kernels (equivalence oracle)
ssgs_native.c # Compiled C kernels (Levinson-Durbin, emission, EM, decode)
ssgs_native.py # ctypes wrapper + benchmark for the C kernels
train_on_folder.py # Folder-based training with checkpointing
demo_enhanced_capabilities.py # State-count showcase
example_checkpoints.py # Checkpoint usage example
example_fidelity.py # Fidelity-focused generation example
test_ssgs.py # Training + generation demo
test_checkpoints.py # Checkpoint save/load tests
test_adaptive_persistence.py # Adaptive learning tests
test_data_indexing.py # Feature index tests
test_gain_control.py # Gain control tests
test_generative_capabilities.py # State-count tests
test_regression_guard.py # Native load + core/pipeline equivalence guard
Makefile # Builds libssgs_native.so
requirements.txt # Dependencies
ssgs_demo.png # Example output
ssgs_analysis.png # Diagnostic output
README.md # This document \
Damien Davison & Michael Maillet & Sacha Davison Recursive AI Devs
We build hybrid-symbolic neural, statistical, and physical AI systems by algorithm decomposition — extracting the useful primitives and recombining them into new model classes.
If you use SSGS in research or production, please cite the authors.
This project is licensed under the Apache License, Version 2.0.
Copyright 2025 Damien Davison & Michael Maillet Recursive AI Devs
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy at:
http://www.apache.org/licenses/LICENSE-2.0
This product includes original work by:
Damien Davison & Michael Maillet & Sacha Davison (Recursive AI Devs)
Additional details can be found in the project's LICENSE and source headers.