PLAN: correct the stale figures, widen rule 1 to the org, restate rule 6

[mmcky]

Documentation-only. No data, no manifests, no migration.yml status changes — the strict audit exits 0 before and after, and audit.json regenerates byte-identically.

These are the corrections an independent review of QuantEcon/workspace-lectures#23 turned up, landed ahead of the high_dim_data fold so the fold is planned against accurate rules.

Seven stale figures

Every number in PLAN.md that restated the audit dashboard's stats had drifted by three repoint sets — set 1 (mpd2020), set 2 (longprices + chapter_3) and set 3 (the five french_rev files) — and each understated progress. All seven were contradicted by the audit.json the document itself cites.

Was	Is
10 of 41 migrated, 31 to go	18 of 41, 23 to go
35 committed orphans	26 — dp 10, programming 5, wasm 5, intro 3, python.myst 2, continuous_time_mcs 1
16 multi-consumer files, 16 of the 31 remaining	8 of the 23
Track A: 17	17, of which 9 remain
manifests for "the 9 files now in lectures/"	21 non-.yml files, 18 manifests; only business_cycle_data.csv lacks one
Phase 9: 31 datasets, 35 orphans	23 and 26

The headline now says where the numbers come from and warns that any figure restating them is a copy that can drift.

Rule 1 — the consumer set is the org

Rule 1 was scoped to lecture-wasm, and in practice has been read as the repos in manifest.yml. That is correct for authoring and wrong for deletion safety: a repo that fetches another repo's committed blobs by URL is a rule-1 consumer regardless of how its content is produced.

Three classes sit outside this repo's eight scanned repos, and each has already been missed once — translations (five of the six live editions read another repo's blobs, and lecture-intro.zh-cn alone holds 7 reads of the high_dim_data six), generated mirrors (lecture-python-intro.notebooks), and course forks and canaries (tom-econ370-2025, test-actions-lecture-intro — both of which are serving 404s today).

The sweep method is now recorded with it: clone and grep, because code search cannot find a bare URL and returns a confident zero — though it does index repo-name tokens, so gh search code 'high_dim_data org:QuantEcon' finds every consumer of that repo including the translations.

Rule 6 — 21 reads across three repos, 12 needing a host change

Three corrections, all of which change what the fold PR has to do.

The count. 14 reads across two repos is 21 across three. lecture-intro.zh-cn holds seven more — heavy_tails.md:810,837,838,862, mle.md:105, inequality.md:256, _static/lecture_specific/inequality/data.ipynb:37 — and is invisible to every audit run, since SCAN_REPOS excludes translations by decision at scripts/build_audit.py:45-46.

Only 12 need the host changed. github.com/<org>/<repo>/raw/… is a smart redirect that routes per path by LFS status, so the two reads already on that form survive an org/repo/path swap with no host decision. The rule previously said "move all 14 off the media host", which is impossible for two of them.

The acceptance grep could never pass. As written it was grep -rn '…' repos/, which matches PLAN.md's own occurrences of the string. Now scoped to the consuming lecture trees, with a note that lecture-intro.zh-cn is never under repos/ and has to be cloned for it.

Also recorded: the media endpoint routes per path, not per repo — measured against high_dim_data's own untracked README.md and cross_section/webscrape_forbes.ipynb, both 404 on media inside an LFS-enabled repo — and the strict audit exits 0 on a fold left entirely on the media host. A media-host URL under a data-lectures path classifies as pattern: data-lectures, so it passes while 404ing at read time. scripts/build_audit.py:196 already computes lfs_media per reference and nothing asserts on it; closing that is the first gate on the fold, tracked separately.

Other corrections

LFS-on-raw is silent, not loud. The sequencing constraint said consumers "fail with a confusing parse error". They do not: the URL returns HTTP 200 with pointer text, and pd.read_csv raises nothing at all — it returns a 2×1 frame whose single column name is version https://git-lfs.github.com/spec/v1. Every affected lecture read is a read_csv, so silence is the case that matters, and a status-code check is a false green. Verify with curl -s <url> | head -1.

Two workflows check this repo out with lfs: true, not one. audit-dashboard.yml:43 (the Pages deploy) and consumed-file-check.yml:22 (every pull request). AGENTS.md asserted lfs: false for the Pages job while the workflow says true, and never mentioned the second. Both must flip before sources/SCF_plus.dta lands — LFS bandwidth is an org-wide quota shared with high_dim_data, and a 403 on LFS downloads takes out the live media-host reads in intro, wasm and zh-cn simultaneously. That is a lecture outage, not a CI failure.

SCF_plus.dta is 0.88% under the hard blob limit — 103,934,093 B against 104,857,600 B. It must stay LFS-tracked permanently; an upstream vintage 1% larger could not be pushed as plain git at all.

generating_mini.md's two to_csv writes are commented out upstream, so the builder as committed produces both frames in memory and writes nothing. Its fetch and transform stages are complete, so this is a small fix at migration time rather than an unrecovered builder.

P3 is reframed. Its stated tests — "served URL makes the raw-vs-media trap invisible" and "Pages handles LFS objects (lfs: true)" — became unrunnable by construction when the Phase 3 storage decision made the published tree 100% plain git. P3 tests the host migration off media.githubusercontent.com, across three consuming repos including one the audit cannot see. It also deletes nothing: neither intro nor wasm holds a copy of the six files, and archiving high_dim_data preserves serving on both hosts, so rule 3's phase 2 does not apply and the set is fully reversible.

Part of QuantEcon/workspace-lectures#23. Upstream: QuantEcon/meta#336.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Updates the repository’s planning and agent guidance documentation to align with current audit-derived counts and to clarify migration/repoint rules (notably org-wide consumer discovery and the media.githubusercontent.com host-migration implications for the high_dim_data fold), plus a small wording clarification in the pending P3 wave description.

Changes:

• Refreshes stale progress figures in PLAN.md and adds guidance to prefer citing audit.json over copied numbers.
• Clarifies repoint rules: expands rule-1 “consumer set” to the whole org and refines rule-6 to distinguish host changes needed for media-host reads.
• Rewords the P3 pending wave scope in migration.yml to reflect the host-migration framing; updates AGENTS.md to note both workflows currently checkout with lfs: true.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
PLAN.md	Updates audit-derived counts and refines repoint/migration rules (org-wide consumer sweep; media-host vs smart-redirect distinction; acceptance grep scoping).
migration.yml	Rephrases the pending P3 wave scope to emphasize host-migration requirements for LFS→plain-git fold cases.
AGENTS.md	Clarifies that two workflows currently use lfs: true and documents the intended lfs toggling policy.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.