Uh oh!
There was an error while loading. Please reload this page.
fix(distill): train from in-memory labels; never re-read the just-written file - #25
Merged
Conversation
added 2 commits
August 22, 2026 13:05
…tten file Arabic tiny labeled all 11,790 srcs, then the post-label re-read of teacher_labels.jsonl served a stale volume replica (2 visible pairs) and the run trained on 2 pairs, saving garbage. The just-generated labels are now kept in memory and used directly; the file is only read on resume runs. The torn-view guard is unconditional, and the label loop gets a final volume commit (the modulo could skip the tail).
The follow-on hole: when the Step-1 resume-read declares everything done, Step 2 re-reads the file — that read can itself serve a stale replica (6 valid pairs of 27,324 persisted), and raising there makes the watchdog relaunch into the identical state forever. On a torn read the run now regenerates all labels in-container and trains from the in-memory rows. Labeling extracted into a label_all() closure shared by the fresh/resume/regenerate paths.
Uh oh!
There was an error while loading. Please reload this page.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan