Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -133,7 +133,7 @@ jobs:
uses: actions/attest-build-provenance@v1
continue-on-error: true
with:
subject-name: github.com/interscript/ml-models
subject-name: github.com/interscript/interscript-ml
subject-digest: sha256:${{ github.sha }}

publish-hf:
Expand Down
10 changes: 5 additions & 5 deletions TODO.distribution/00-overview.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -18,23 +18,23 @@ zero manual steps after a `git tag`.
## Channel architecture (hybrid, MECE)

```
Source code → github.com/interscript/ml-models
Source code → github.com/interscript/interscript-ml
(this repo: framework, configs, tests, CI/CD,
release changelog, issue tracking)

Trained weights → huggingface.co/interscript/<task_name>
(PyTorch, full) (model card, datasets, transformers.js hook,
free Cloudflare-backed CDN, researcher entry)

Browser-ready ONNX → github.com/interscript/ml-models/releases/tag/<task>-v<X.Y.Z>
Browser-ready ONNX → github.com/interscript/interscript-ml/releases/tag/<task>-v<X.Y.Z>
(one .onnx asset per release; checksums attached;
stable immutable URLs; programmatic via `gh`)

CDN mirror → jsdelivr.net/gh/interscript/ml-models@<tag>/...
CDN mirror → jsdelivr.net/gh/interscript/interscript-ml@<tag>/...
(global edge cache of the GH release assets;
CORS-friendly; no rate limits for end users)

JS glue package → npm: @interscript/models
JS glue package → npm: npm `secryst` (manifest now the models.yaml index)
(tiny manifest of current versions per task;
consumed by interscript-ts to resolve URLs)

Expand All@@ -59,7 +59,7 @@ Each channel serves a real audience. None is redundant.
- `01-github-releases.md` — release asset layout, tag conventions, asset naming
- `02-huggingface-hub.md` — HF org, model cards, datasets, auto-conversion
- `03-cdn-strategy.md` — jsdelivr + GH releases URL convention, fallback chain
- `04-npm-packages.md` — `@interscript/models` manifest package
- `04-npm-packages.md` — `npm `secryst` (manifest now the models.yaml index)` manifest package
- `05-ruby-model-cache.md` — gem-side download + verify + atomic write
- `06-versioning-scheme.md` — per-task semver, breaking-change rules
- `07-ci-cd.md` — `.github/workflows/release.yml` full design
Expand Down
2 changes: 1 addition & 1 deletion TODO.distribution/01-github-releases.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -57,7 +57,7 @@ Asset filenames follow `<task>-<variant>.<ext>`:
The CDN URL is computable from `<task>` + `<version>` + `<variant>`:

```
https://github.com/interscript/ml-models/releases/download/<task>-v<ver>/<task>-<variant>.onnx
https://github.com/interscript/interscript-ml/releases/download/<task>-v<ver>/<task>-<variant>.onnx
```

No central manifest needed to compute URLs. The convention IS the API.
Expand Down
12 changes: 6 additions & 6 deletions TODO.distribution/03-cdn-strategy.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,12 +13,12 @@ Zero cost to us.

Primary (CDN-cached):
```
https://cdn.jsdelivr.net/gh/interscript/ml-models@<task>-v<x.y.z>/<task>.onnx
https://cdn.jsdelivr.net/gh/interscript/interscript-ml@<task>-v<x.y.z>/<task>.onnx
```

Fallback (direct GH):
```
https://github.com/interscript/ml-models/releases/download/<task>-v<x.y.z>/<task>.onnx
https://github.com/interscript/interscript-ml/releases/download/<task>-v<x.y.z>/<task>.onnx
```

jsdelivr mirrors GitHub Releases content via the `@<tag>` syntax. The
Expand DownExpand Up@@ -89,19 +89,19 @@ Result: second visit loads instantly. Models cached indefinitely
becomes:

```typescript
const DEFAULT_MODEL_BASE = "https://cdn.jsdelivr.net/gh/interscript/ml-models@"
const DEFAULT_MODEL_BASE = "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@"
```

`loadModel({ kind: "rababa", id: "default" })` resolves to:
1. Look up `<task>` from id (e.g. `"default"` → `"rababa_arabic"`)
2. Look up `<version>` from `@interscript/models` npm package
2. Look up `<version>` from `npm `secryst` (manifest now the models.yaml index)` npm package
3. Build URL: `${DEFAULT_MODEL_BASE}<task>-v<version>/<task>.onnx`
4. Fetch with fallback chain
5. Cache in IndexedDB

## Version manifest (npm)

The `@interscript/models` npm package exposes a tiny JSON manifest:
The `npm `secryst` (manifest now the models.yaml index)` npm package exposes a tiny JSON manifest:

```json
{
Expand All@@ -114,7 +114,7 @@ The `@interscript/models` npm package exposes a tiny JSON manifest:
`interscript-ts` reads this at startup. Users can pin a version via:

```typescript
setModelBase("https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/")
setModelBase("https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/")
```

## Acceptance
Expand Down
26 changes: 13 additions & 13 deletions TODO.distribution/04-npm-packages.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,7 +22,7 @@ go live. Adds:
- `transliterateAsync()` for ML funcalls
- Model provisioning layer

### 2. `@interscript/models` (new, tiny)
### 2. `npm `secryst` (manifest now the models.yaml index)` (new, tiny)

A pure-JSON manifest package. No JS code. Just a version index:

Expand All@@ -34,8 +34,8 @@ A pure-JSON manifest package. No JS code. Just a version index:
"version": "1.0.0",
"size_bytes": 6197600,
"sha256": "3a7f2b...",
"url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx",
"vocab_url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic-vocab.json"
"url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx",
"vocab_url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic-vocab.json"
},
"rababa_hebrew": { ... },
"secryst_thai_ipa": { ... }
Expand DownExpand Up@@ -72,8 +72,8 @@ Same pattern for `@interscript/model-rababa-hebrew`,

```
1. CI builds ONNX, attaches to GH release
2. CI updates @interscript/models manifest
3. CI runs: npm version patch && npm publish for @interscript/models
2. CI updates npm `secryst` (manifest now the models.yaml index) manifest
3. CI runs: npm version patch && npm publish for npm `secryst` (manifest now the models.yaml index)
4. Users running `interscript-ts@0.2.x` see the new version on next
app restart (model cache invalidates by URL change)
5. Users running `@interscript/model-rababa-arabic` opt-in via
Expand All@@ -82,16 +82,16 @@ Same pattern for `@interscript/model-rababa-hebrew`,

## Why three packages (not one)

- **MECE.** Runtime code (interscript-ts) vs version manifest (@interscript/models) vs bundled binary (@interscript/model-*). Three concerns, three packages.
- **Bundle size.** Browser apps want interscript-ts + @interscript/models (small). They fetch ONNX at runtime from CDN. Bundle stays small.
- **MECE.** Runtime code (interscript-ts) vs version manifest (npm `secryst` (manifest now the models.yaml index)) vs bundled binary (@interscript/model-*). Three concerns, three packages.
- **Bundle size.** Browser apps want interscript-ts + npm `secryst` (manifest now the models.yaml index) (small). They fetch ONNX at runtime from CDN. Bundle stays small.
- **Air-gap friendliness.** Enterprises install @interscript/model-* via private npm mirror. No CDN calls.
- **Versioning independence.** Runtime API breaks != model version bumps != model retrain.

## Why `@interscript/models` not embedded in `interscript-ts`
## Why `npm `secryst` (manifest now the models.yaml index)` not embedded in `interscript-ts`

If manifest lives in `interscript-ts`, every model release forces a
runtime bump. Decouples release cadence:
- Model retrain: bump `@interscript/models` only.
- Model retrain: bump `npm `secryst` (manifest now the models.yaml index)` only.
- Runtime API change: bump `interscript-ts` only.

## Install ergonomics
Expand All@@ -104,12 +104,12 @@ npm install interscript-ts
npm install interscript-ts @interscript/model-rababa-arabic

# Pin to specific model version:
npm install @interscript/models@1.2.0
npm install npm `secryst` (manifest now the models.yaml index)@1.2.0
```

## Versioning

`@interscript/models` uses CalVer-style: `1.<month>.0` so users can
`npm `secryst` (manifest now the models.yaml index)` uses CalVer-style: `1.<month>.0` so users can
see at a glance when the manifest was last updated. (e.g. `1.8.0` =
August 2026 release.)

Expand All@@ -118,7 +118,7 @@ August 2026 release.)

## Acceptance

- [ ] `@interscript/models` published (placeholder manifest)
- [ ] `npm `secryst` (manifest now the models.yaml index)` published (placeholder manifest)
- [ ] `@interscript/model-rababa-arabic` published (placeholder)
- [ ] `interscript-ts@0.2.0` released depending on `@interscript/models`
- [ ] `interscript-ts@0.2.0` released depending on `npm `secryst` (manifest now the models.yaml index)`
- [ ] README documents install patterns for all three combos
2 changes: 1 addition & 1 deletion TODO.distribution/05-ruby-model-cache.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -74,7 +74,7 @@ rababa_arabic:
version: "1.0.0"
sha256: "3a7f2b..."
size_bytes: 6197600
url_base: "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0"
url_base: "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0"
```

Override at runtime:
Expand Down
6 changes: 3 additions & 3 deletions TODO.distribution/06-versioning-scheme.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -65,8 +65,8 @@ Following SemVer pre-release syntax:
Pre-releases are:
- Uploaded to GH Releases as drafts
- Published to HF Hub with `stage: prerelease` tag
- **NOT** added to `@interscript/models` manifest until promoted
- Available via explicit URL: `cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0-rc.1/`
- **NOT** added to `npm `secryst` (manifest now the models.yaml index)` manifest until promoted
- Available via explicit URL: `cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0-rc.1/`

## Versioning the framework itself

Expand All@@ -79,7 +79,7 @@ Framework: `0.1.0` → `0.2.0` → ... → `1.0.0` (after first production task)

When a task version is superseded:
1. New release tag pushes normally.
2. `@interscript/models` manifest points `default` → new version.
2. `npm `secryst` (manifest now the models.yaml index)` manifest points `default` → new version.
3. Old version stays in GH Releases (immutable).
4. Old version's HF model card gets a banner: "Superseded by v1.1.0".
5. `Interscript.clear_cache!(older_than: 90.days)` eventually prunes.
Expand Down
2 changes: 1 addition & 1 deletion TODO.distribution/07-ci-cd.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,7 +11,7 @@
3. Benchmarks (DER/PER, latency)
4. GH Release with auto-generated notes
5. HF Hub upload (PyTorch + ONNX + card)
6. `@interscript/models` npm manifest bump
6. `npm `secryst` (manifest now the models.yaml index)` npm manifest bump
7. (Optional) `@interscript/model-<task>` npm publish

Zero manual steps after the tag.
Expand Down
16 changes: 8 additions & 8 deletions TODO.distribution/08-supply-chain.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -60,12 +60,12 @@ Every release asset has `<asset>.provenance.json`:
"runner": "ubuntu-latest"
},
"source": {
"repository": "github.com/interscript/ml-models",
"repository": "github.com/interscript/interscript-ml",
"ref": "refs/tags/rababa_arabic-v1.0.0",
"commit_sha": "abc123def456..."
},
"materials": [
{ "uri": "git+github.com/interscript/ml-models", "digest": { "sha1": "abc123..." } },
{ "uri": "git+github.com/interscript/interscript-ml", "digest": { "sha1": "abc123..." } },
{ "uri": "pypi://torch@2.2.0", "digest": { "sha256": "..." } },
{ "uri": "pypi://transformers@4.42.0", "digest": { "sha256": "..." } }
],
Expand All@@ -84,7 +84,7 @@ Consumers can verify with `gh attestation verify`:

```bash
gh attestation verify rababa_arabic.onnx \
--repo interscript/ml-models \
--repo interscript/interscript-ml \
--predicate-type https://slsa.dev/provenance/v1
```

Expand DownExpand Up@@ -114,16 +114,16 @@ private key to manage (keyless signing via short-lived cert).

## Manifest package ties it together

`@interscript/models` manifest carries all three:
`npm `secryst` (manifest now the models.yaml index)` manifest carries all three:

```json
{
"rababa_arabic": {
"version": "1.0.0",
"url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx",
"url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx",
"sha256": "3a7f2b...",
"provenance_url": "https://github.com/interscript/ml-models/attestations/...",
"sigstore_bundle_url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx.bundle"
"provenance_url": "https://github.com/interscript/interscript-ml/attestations/...",
"sigstore_bundle_url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx.bundle"
}
}
```
Expand All@@ -148,7 +148,7 @@ Three threats, three mitigations. MECE.

If our signing identity is compromised:
1. Revoke the cert via Sigstore Rekor entry (public log).
2. Yank the manifest entry (publish new `@interscript/models`).
2. Yank the manifest entry (publish new `npm `secryst` (manifest now the models.yaml index)`).
3. Cut new release with new signing identity.
4. Announce via RSS / mailing list / model card banner.

Expand Down
4 changes: 2 additions & 2 deletions TODO.distribution/09-model-cards.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -110,7 +110,7 @@ Interscript.transliterate("<map_code>", "<input>")
### Direct ONNX (any language)

\`\`\`bash
curl -LO https://github.com/interscript/ml-models/releases/download/<task>-v<VERSION>/<task>.onnx
curl -LO https://github.com/interscript/interscript-ml/releases/download/<task>-v<VERSION>/<task>.onnx
# Use onnxruntime to load + run
\`\`\`

Expand All@@ -119,7 +119,7 @@ curl -LO https://github.com/interscript/ml-models/releases/download/<task>-v<VER
\`\`\`bibtex
@software{interscript_<task>,
title = {Interscript <task_full_name>},
url = {https://github.com/interscript/ml-models},
url = {https://github.com/interscript/interscript-ml},
version = {<VERSION>}
}
\`\`\`
Expand Down
6 changes: 3 additions & 3 deletions TODO.distribution/14-sunset-kill-switch.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,7 +22,7 @@ across every consumer — without breaking anyone's app.

## Manifest as the kill switch

`@interscript/models` manifest is the source of truth for "what's the current version". Pull the entry, every consumer sees the retraction.
`npm `secryst` (manifest now the models.yaml index)` manifest is the source of truth for "what's the current version". Pull the entry, every consumer sees the retraction.

```json
{
Expand All@@ -46,7 +46,7 @@ across every consumer — without breaking anyone's app.

A retraction script `scripts/retract.sh <task> <version>` performs:

1. **Manifest bump** — publish new `@interscript/models` with `status: retracted`.
1. **Manifest bump** — publish new `npm `secryst` (manifest now the models.yaml index)` with `status: retracted`.
2. **HF Hub** — add `retracted: true` to model card metadata, pin a banner.
3. **GH Release** — convert to "draft" (preserves asset URLs but hides from listing).
4. **Slack / mailing list** — automated announcement.
Expand DownExpand Up@@ -81,7 +81,7 @@ Building this on day one (even if we never need it) is far cheaper than retrofit

## Acceptance

- [ ] `@interscript/models` manifest carries `status` field
- [ ] `npm `secryst` (manifest now the models.yaml index)` manifest carries `status` field
- [ ] `scripts/retract.sh` and `scripts/restore.sh` work end-to-end
- [ ] interscript-ts warns on startup if cached version is retracted
- [ ] Game-day exercise: simulate a retraction, verify consumers respond
11 changes: 11 additions & 0 deletions TODO.distribution/README.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
# Distribution planning notes (historical)

Written 2026-08-14/15, before the 2026-08-20 interscript × secryst
restructure. The repo and package names inside were mechanically
updated to the current ones on 2026-08-22 (`interscript/ml-models` →
`interscript/interscript-ml`, `@interscript/models` → the models.yaml
index consumed by the `secryst` crystals, `interscript-js` →
`interscript-ts`, `interscript-python` → `interscript-py`); the
scheme itself predates IMF v1 and the parts contract, so treat these
as design history, not the current release process. The live process
is `scripts/publish_model.py` + `docs/imf-v1.md`.
48 changes: 47 additions & 1 deletion docs/RESULTS.md
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
# ml-models results
# interscript-ml results

Evaluated results for models produced in this repository. Each section
anchor is the provenance target referenced by model metadata
Expand DownExpand Up@@ -59,3 +59,49 @@ IMF test split (1,864 long sentences), same harness for both models

Shrink cost +5.58pp — inside the ~5.6pp budget pre-accepted for this
pair (rababa docs/DISTILL-SOURCE-PROMPT.md section 2).

## tha-g2p-small-1.0 — Thai G2P client tier (2026-08-22)

The client-tier release of the Thai G2P distillation: run-003,
ByT5-small student on the full label set (48,757 usable beam-4 labels
from the B-K/umt5-thai-g2p-v2-0.5k teacher). Same harness as
tha-g2p-base-1.0 (beam-4, corpus-level PER, 1,219 held-out Kaikki Thai
test sentences, `src/gpu/modal_distill.py::evaluate_per`; checkpoint
re-measured 2026-08-22 for this release).

| Model | PER | Exact match |
|---|---|---|
| Teacher (B-K/umt5 hub base) | 4.43% | 95.57% |
| **Student (ByT5-small, client rung)** | **12.06%** | 87.94% |

Shrink cost +7.63pp — outside the +5pp server-tier gate (that gate is
met by tha-g2p-base-1.0 at 9.19%): shipped anyway per the frontier
below, as the smallest artifact that does not collapse. Exported at
int8 (~300MB); see the frontier table for why no smaller rung exists
today.

## Client-tier size–quality frontier (2026-08-22)

Thai G2P, same harness (beam-4 corpus PER, 1,219 Kaikki sentences; teacher
B-K umt5 4.43%):

| Student | Init | Params | Artifact (int8) | PER |
|---|---|---|---|---|
| custom 8+8 d384 | random | 33M | ~30MB | 75.80 (collapsed) |
| custom 8+8 d384 + bridges | random | 33M | ~30MB | 71.12 |
| custom 10+10 d512 + bridges | random | 70M | ~70MB | 78.51 |
| ByT5-small | pretrained | 300M | ~300MB | 12.06 |
| ByT5-base (server tier) | pretrained | 580M | 1.2GB fp32 | 9.19 |

Findings: (1) random-init byte-level seq2seq collapses regardless of
capacity at this scale — the microkimi bridges improve structure (75.8 →
71.1) but cannot rescue G2P accuracy; enlarging without pretraining does
not help (70M = 78.5). (2) ByT5-small's width (d=1472) dominates its
parameter count — depth-pruning yields no useful intermediate rung
(263M). (3) The pretrained rung is the whole quality cliff: 300M at
12.06% (run-003, full labels; 12.63% on the 23K subset) vs 70M at 78.5%.

Conclusion: G2P client tier ships at the ByT5-small rung (~300MB int8)
today; a 30–70MB G2P tier requires byte-level pretraining of the small
model first (future work). Copy-task languages (Arabic/Hebrew
diacritization) have a different viability and are evaluated separately.
2 changes: 1 addition & 1 deletion docs/imf-v1.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -51,7 +51,7 @@ model.zip
| `license` | str | non-empty (strict gate) |
| `trained_from` | str | repo + run/checkpoint id |
| `metrics` | list | `{name, value, protocol, source}`; `source` must be a `RESULTS.md#anchor` (strict gate) |
| `parity` | map? | `{samples, cer_delta}`; strict gate: samples >= 500, cer_delta <= 0.2pp |
| `parity` | map? | `{samples, cer_delta}`; strict gate: samples >= 500, cer_delta <= 0.2pp fp32 / 1.0pp fp16 / 2.0pp int8 |
| `sha256` | map | every `*.onnx` member -> hex digest; no dangling entries |

The `id` does not encode precision: `khm-latn-1.0-fp16.zip` and
Expand Down
Loading
Loading