fix(chatterbox-s3g): reset Metal scheduler between decodes by jhen0409 · Pull Request #1 · mybigday/codec.cpp

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.

Tests run

cmake -S . -B build -DGGML_METAL=ON -DCODEC_BUILD_TTS_CLI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 8
python3 -m py_compile tests/e2e/chatterbox_s3g_decode_smoke.py
python3 tests/e2e/chatterbox_s3g_decode_smoke.py (asset-gated skip: models/chatterbox is not present in this checkout)
git diff --check

jhen0409 · 2026-08-09T08:01:17Z

Summary

release scheduler graph state before freeing the eval context it points into
fully recreate the scheduler after Chatterbox S3G decode graphs, including clearing the reserved graph size so the next decode provisions a fresh scheduler
exercise two GPU-offloaded decodes through the same codec context in the S3G smoke test

Root cause

codec_graph_release freed eval_ctx before releasing scheduler graph state, even though the scheduler retains tensor pointers into that graph. Resetting the scheduler is sufficient for the other graph kinds, but Metal can retain a stale allocation for Chatterbox S3G's unusually large unified decode graph. The next decode through the same context then completes with all-zero PCM.

The scheduler is now reset while the eval graph is still alive. For CODEC_GRAPH_CHATTERBOX_S3G_DECODE, it is fully freed and sched_reserved_graph_size is cleared so codec_sched_ensure_capacity recreates it on the next decode.

Validation evidence

Same-context Metal runner before this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0 peak=0

After this fix:

run 1: tokens=44 samples=42240 rms=0.104253 peak=0.814765
run 2: tokens=44 samples=42240 rms=0.104253 peak=0.814765

The updated smoke test now loads with GPU offload and rejects silence or output drift across two decodes on one context.

Scope and performance

Other allocated graph kinds keep their existing scheduler and only reset its graph state, now in the safe order. Chatterbox S3G decode graphs recreate the scheduler after release on every backend so the Metal-specific stale allocation cannot survive. That adds scheduler creation/allocation planning to the next S3G decode, but does not reload the model or its weights and does not affect other graph kinds.