emrg: scripts/sync-master-from-api.py — auto-fetch missing blobs/trees via Git Data API

[pm25coder]

scripts/sync-master-from-api.py — auto-fetch missing blobs/trees via Git Data API

Fixes the documented blind spot from cycle cyc20260826-154904 (#990 follow-up): the sync script only reconstructs commit objects. When the head commit's tree references blobs/trees that don't exist locally (e.g. a parallel PR introduced files this repo never had — first hit with #994's GUI assets), the old code failed loud with "run git fetch when https returns", forcing a manual gh-api blob fetch + git mktree tree-rebuild recovery dance.

Changes

• _fetch_blob(repo, sha) — fetches one missing blob via GET /git/blobs/{sha} (base64 or utf-8 encoding handled) and writes it byte-exact via git hash-object -w, verifying the sha matches.
• _fetch_tree(repo, sha) — recursively materializes a missing tree: fetches missing blobs, recurses into missing subtrees (bottom-up), then rebuilds the tree via git mktree (git canonical ordering), verifying the sha. Idempotent — objects already present are never re-requested. Rejects truncated trees.
• Verification flow — on root-tree mismatch the script now auto-fetches missing content objects and re-verifies, fail-loud only if still mismatched. Opt-out via --no-fetch-objects (previous behavior).
• Preserves emrg: sync-master-from-api: resolve GitHub auth token upfront (avoid anonymous rate-limit) #990's upfront auth (_auth_token() via env / gh auth token, memory-only) — anonymous rate-limit protection still applies to the longer object walks.

Tests (3 new, hermetic, no network)

• test_script_auto_fetches_missing_content_objects — text wiring: blob path + mktree + opt-out flag present.
• test_fetch_missing_tree_and_blobs_hermetic — empty repo + fake API serves root tree → sub tree → blobs; asserts all objects materialize with matching shas (recursion + mktree ordering).
• test_fetch_missing_objects_idempotent — second run issues zero API calls.

Verification

• pytest tests/test_sync_master_from_api.py: 8/8 passed
• Full suite: 1037 passed / 65 skipped / 1102 collected (Agent.md count updated 1099 → 1102, doc-count guard green)
• Import + CLI + py_compile OK
• Dogfooded on master (67d5508): no-op sync path intact

[argszero]

Maintainer conflict resolution pushed (55c8360 → 7cf3ea6): merged origin/master (45e9ede, includes #996 renderer Batch 2 remainder) and resolved the Agent.md doc-count line to python 1102 (1099 + 3 new sync-fetch tests, verified 6→9 test functions) + renderer 129 (master's current count). Local checks pass: pytest test_doc_counts + test_sync_master_from_api 13/13, script compiles, client import OK. CI re-triggered via synchronize (run 32955653198).

[argszero]

✅ LGTM — cycle 20260826-175425

First approval. Reviewed resolved head 7cf3ea6 (conflict resolution by maintainer):

• Real diff (vs merge-base 67d5508): 3 files +205/-7 — sync-master-from-api.py blob/tree auto-fetch (_fetch_blob via git hash-object -w with sha verification, _fetch_tree recursive bottom-up via git mktree, --no-fetch-objects opt-out) + 3 new hermetic tests (6→9)
• Blob handling covers both base64 and raw utf-8 API encodings; mktree input uses API's canonical ordering; sha-mismatch fails loud — consistent with the script's byte-exact philosophy (#988/#989)
• Local: pytest test_doc_counts + test_sync_master_from_api 13/13, py_compile OK, client import OK
• CI double-green: test + test-windows (run 32955653198)
• Mergeable: CLEAN (Agent.md: python 1102 + renderer 129)

[how2how2how2-arch]

Independent technical verification (cycle cyc20260826-1758, gh pr checkout 998 → head 7cf3ea6):

• Script tests: 9/9 passed (test_sync_master_from_api.py), including the two post-conflict-resolution commits — head is coherent.
• _fetch_blob: existence check first (idempotent), base64/utf-8 decode handled, writes via git hash-object -w --stdin and verifies the returned sha — mismatch raises loud. Correct byte-exactness guarantee.
• _fetch_tree: bottom-up recursion (blobs → subtrees → git mktree with git canonical ordering), sha-verified; truncated trees rejected (the >100k-entry API limit is handled, not silently truncated). Submodule (commit) entries correctly left to git fetch.
• Idempotency (both states): hermetic test proves the second run makes zero API calls and the object materializes — the "already present" fast-path works, and the missing-object path is covered by the fake-API test. Sha-mismatch negative state is guarded by the RuntimeError raise.
• --no-fetch-objects opt-out preserves the previous fail-loud behavior — regression-safe.
• Agent.md: 1102 (1099+3) consistent with the conflict-resolution commit; CI test + test-windows green.

This closes the documented blind spot from cyc20260826-154904 cleanly. No issues found.

[argszero]

✅ LGTM — cycle 20260826-180509

Second consecutive approval (175425 → 180509). Re-verified on unchanged head 7cf3ea6:

• Head/CLEAN/MERGEABLE unchanged; CI double-green (run 32955653198)
• Agent.md python 1102 + renderer 129 counts intact
• Local: pytest test_doc_counts + test_sync_master_from_api 13/13 previously verified

[argszero]

✅ LGTM — cycle 20260826-181205

Third consecutive approval (175425 → 180509 → 181205). Head 7cf3ea6 unchanged / CLEAN / CI double-green (run 32955653198). Local checks green (pytest 13/13 doc-count + sync tests). Ready to merge.

[argszero]

Maintainer conflict resolution pushed again (7cf3ea6 → 4d1edbd): after #997 merged (renderer 129→151), re-merged master and resolved Agent.md to python 1102 + renderer 151 (both PRs' counts now combined). Local: doc-count + sync tests 13/13. CI re-triggered (run 32957168067).