Skip to content

release: tha-g2p-small-1.0#24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 2 commits into from
Aug 22, 2026
Merged

release: tha-g2p-small-1.0 #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6e16982
release: tha-g2p-small-1.0 (int8, parity cer_delta 0.0803pp on 1219 s…
Aug 22, 2026
8b300b5
metrics: tha-g2p-small ref -> main (section landed via PR #23)
Aug 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions models.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -93,3 +93,16 @@ models:
- {name: per_student, value: 9.19, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
parity: {samples: 1219, cer_delta: 0.0}
license: BSD-3-Clause
tha-g2p-small-1.0:
task: g2p
scripts: [Thai, IPA]
precision: int8
filename: tha-g2p-small-1.0-int8.zip
url: https://github.com/interscript/interscript-ml/releases/download/tha-g2p-small-1.0/tha-g2p-small-1.0-int8.zip
sha256: f2cfa93a42d98aabf08fdc82dfc6bd1574e21993f36558aa581fbe29e0588a94
size: 257468517
metrics:
- {name: per_teacher, value: 4.43, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0}
- {name: per_student, value: 12.06, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0}
parity: {samples: 1219, cer_delta: 0.0803}
license: BSD-3-Clause
2 changes: 1 addition & 1 deletion models/metrics-sources.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -69,7 +69,7 @@ heb-diac-small-1.0:
- {row: "Student (ByT5-small, gate)", column: DER, as: der_student_greedy}
tha-g2p-small-1.0:
repo: interscript/interscript-ml
ref: release/tha-g2p-small-1.0
ref: main
path: docs/RESULTS.md
anchor: tha-g2p-small-10-thai-g2p-client-tier-2026-08-22
protocol: "beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece decode); 1,219 held-out Kaikki Thai test sentences"
Expand Down
53 changes: 28 additions & 25 deletions models/tha-g2p-small/tha-g2p-small-1.0.metadata.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -6,30 +6,33 @@ target: IPA
tokenizer: bytes
opset: 14
decoder: kv
precision: fp32
precision: int8
license: BSD-3-Clause
trained_from: >-
sequence-level KD from the B-K/umt5-thai-g2p-v2-0.5k teacher (4.43%
PER on this harness; the secryst-saved umt5 artifacts are unusable —
transformers 5.15 dropped the untied lm_head); 48,757 usable beam-4
teacher-generated labels (51,321 raw, deduplicated, degenerate outputs
filtered); ByT5-small init google/byt5-small; checkpoint
secryst-checkpoints:/secryst_thai_g2p_distill_small/run-003/best.
Client tier: capacity-limited at +7.63pp over the teacher — the +5pp
gate is met by tha-g2p-base-1.0; this is the smallest rung that does
not collapse (docs/RESULTS.md frontier table, ~300MB int8).
trained_from: 'sequence-level KD from the B-K/umt5-thai-g2p-v2-0.5k teacher (4.43%
PER on this harness; the secryst-saved umt5 artifacts are unusable — transformers
5.15 dropped the untied lm_head); 48,757 usable beam-4 teacher-generated labels
(51,321 raw, deduplicated, degenerate outputs filtered); ByT5-small init google/byt5-small;
checkpoint secryst-checkpoints:/secryst_thai_g2p_distill_small/run-003/best. Client
tier: capacity-limited at +7.63pp over the teacher — the +5pp gate is met by tha-g2p-base-1.0;
this is the smallest rung that does not collapse (docs/RESULTS.md frontier table,
~300MB int8).'
metrics:
- name: per_teacher
value: 4.43
protocol: >-
beam-4, corpus-level PER (total_ed/total_gold over chars of
joined-piece decode); 1,219 Kaikki Thai test sentences; B-K/
umt5-thai-g2p-v2-0.5k teacher; exact match 95.57%
source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0
- name: per_student
value: 12.06
protocol: >-
beam-4, corpus-level PER, same harness as the teacher (src/gpu/
modal_distill.py::evaluate_per, run-003 checkpoint re-measured
2026-08-22); exact match 87.94%; +7.63pp client-tier shrink cost
source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0
- name: per_teacher
value: 4.43
protocol: beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece
decode); 1,219 Kaikki Thai test sentences; B-K/ umt5-thai-g2p-v2-0.5k teacher;
exact match 95.57%
source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0
- name: per_student
value: 12.06
protocol: beam-4, corpus-level PER, same harness as the teacher (src/gpu/ modal_distill.py::evaluate_per,
run-003 checkpoint re-measured 2026-08-22); exact match 87.94%; +7.63pp client-tier
shrink cost
source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-small-1.0
parity:
samples: 1219
cer_delta: 0.0803
sha256:
decoder-kv.onnx: b8d67ecbec9f4a29beafd3e6d431422f084c703a35467d836e81bdb8ceff4cba
decoder.onnx: 81b3580d1d26f78e2aa1786218f50ebe970b23309386fa06bb73b0f27333c0a8
encoder.onnx: 53416ec4c14ede50d2a5b3cd4b0de73c2b7c62e744a7d48d60e12a81ee89244d
Loading