From b38f415fd5f713ad993315ce04b57716235ec2af Mon Sep 17 00:00:00 2001 From: codeitlikemiley Date: Mon, 24 Aug 2026 08:16:46 +0800 Subject: [PATCH] =?UTF-8?q?docs(19):=20=C2=A719.6=20describes=20a=20layout?= =?UTF-8?q?=20reality=20diverged=20from=20=E2=80=94=20say=20where=20it=20w?= =?UTF-8?q?ent?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The diagram plans five things under `training/`. Two exist, two are unwritten, and one was built in Rust instead. Four empty directories would have satisfied the diagram and told a reader nothing, so they are deliberately not created. `datasets/` is the interesting one: the transcript miner shipped at M19.4 as `panday_harness::mining` + `cargo xtask mine`, not as Python. That is the right place for it — it parses panday's own JSONL event logs, whose shape is a Rust type with golden fixtures, and a Python re-implementation would be a second parser for a moving format that disagrees the first time an event gains a field. `export/`'s signing half is also Rust (`panday models sign`). `recipes/` and `registry.py` are unwritten and blocked on a base-model choice and GPU hours; committing unsloth hyperparameters for a model nobody has picked would be config-shaped fiction. ADR-001's boundary holds — Rust serves, Python trains — the line just sits further into Python than the diagram drew it. Claude-Session: https://claude.ai/code/session_017kFpYDqvz6sKGSkM4YKaRf --- docs/19-training.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/19-training.md b/docs/19-training.md index 5f5c733..efc0c14 100644 --- a/docs/19-training.md +++ b/docs/19-training.md @@ -160,6 +160,24 @@ training/ # Python uv project — the ONE Python island (ADR- └── registry.py # artifact registry client → catalog entries (18) ``` +**What is actually here, because the plan above is one of four and the divergence is the +interesting part.** Four empty directories would satisfy this diagram and tell a reader nothing; +they are deliberately not created. + +| Planned | Reality | +|---|---| +| `datasets/` | **Built, in Rust.** `panday_harness::mining` + `cargo xtask mine --logs --out ` (M19.4). It reads panday's own JSONL event logs, whose shape is a Rust type with golden fixtures; a Python re-implementation would be a second parser for a format that moves, and the two would disagree the first time an event gained a field. The consent flags, PII scrub and provenance the milestone asks for live there. | +| `evals/` | **As planned** — `training/evals/json_discipline.py`, inspect-ai through the gateway. | +| `recipes/` | **Unwritten.** unsloth/trl configs are per base model, and no base model is chosen. Writing them now would be committing hyperparameters for a model nobody has picked, on hardware nobody has rented. | +| `export/` | **Unwritten, except signing, which is in Rust.** `panday models sign` signs the model index (`sign_index`), so the trust half of the pipeline exists. Merge / GGUF convert / imatrix / quantize all need a trained model on disk. | +| `registry.py` | **Unwritten.** The catalog it would write to is `crates/panday-router/catalog/*.yaml`, read by Rust, and the signed index is already Rust's. Whether a Python client is the right shape is a question to answer when there is an artifact to register — not before. | + +The pattern worth naming: **the deterministic, format-owning half migrated to Rust and the +model-shaped half is waiting on a model.** ADR-001's boundary ("Rust serves, Python trains") still +holds; the line simply sits further into Python than this diagram drew it. Anything above marked +unwritten is blocked on a base-model choice and GPU hours, and none of it is faked in the +meantime. + Format: OpenAI-style `messages` JSONL everywhere (tool-call turns preserved natively — never flattened to text). Dataset builders emit provenance + license fields per example; `semhash` dedup + n-gram decontam runs in the