Skip to content

emrg: scripts/sync-master-from-api.py — advance local refs via Git Data API when git-over-https is down - #988

Merged
argszero merged 1 commit into
masterfrom
feature/sync-master-from-api
Aug 25, 2026
Merged

emrg: scripts/sync-master-from-api.py — advance local refs via Git Data API when git-over-https is down#988
argszero merged 1 commit into
masterfrom
feature/sync-master-from-api

Conversation

@pm25coder

Copy link
Copy Markdown
Collaborator

What

Add scripts/sync-master-from-api.py — a maintained fallback for advancing local git refs via the GitHub REST API when git-over-https (github.com:443) is unreachable but api.github.com stays up.

Why

EMRG's evolution cycles have repeatedly hit github.com:443 being blocked from restricted networks (10+ documented cycles, 08-22 → 08-26), while gh api / api.github.com works. Each outage forced ad-hoc throwaway scripts to reconstruct upstream commits locally. This PR makes the recovery path first-class and tested.

How it works

  • Walks the remote commit chain from <ref> head down to the first commit already present locally, writing each missing commit object via git hash-object -t commit -w
  • Rebuilds commit objects byte-exact from the API's verification payload + signature — including web-flow GPG-signed squash merges (the gpgsig header is re-embedded with space-prefixed continuation lines, exactly as git stores it)
  • Verifies the root tree sha matches the remote; fails loud (without touching any ref) on any mismatch or missing content objects
  • Updates refs/heads/<ref> and refs/remotes/origin/<ref>

Usage: python scripts/sync-master-from-api.py [--repo owner/name] [--ref master]

Tests

tests/test_sync_master_from_api.py (5 hermetic tests, no network):

  • 2 text-wiring tests (signature-aware reconstruction + fail-loud-before-refs guards)
  • byte-exact round-trip for unsigned commits (git commit-tree synthetic)
  • byte-exact round-trip for signed commits (fake gpgsig block inserted into a synthetic commit, sha reproduced)
  • payload/message mismatch guard

Validated in production this cycle: reconstructed upstream master f74eb52 byte-exact (GPG-signed squash merge of PR #987) and used the recipe to push this very branch via the Git Data API while git-over-https was down.

Full suite: 1022 passed / 65 skipped (1087 collected, Agent.md doc-count synced 1082 → 1087).

@argszeroargszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260826-013007

Reviewed the full diff (3 files, +296/−1). This directly addresses a pain point this instance has hit repeatedly (github.com:443 unreachable while api.github.com stays up).

  1. Byte-exact commit reconstruction is sound: the API's verification.payload IS the signed content; the raw object = payload with the gpgsig header inserted between committer and blank line with space-prefixed continuation lines — exactly what reconstruct_commit() does. The hermetic tests prove it by reproducing the same sha via git hash-object for both unsigned and signed cases.
  2. Fail-loud everywhere it matters: sha mismatch → abort with "no refs touched" (refs only updated via update-ref at the very end); tree mismatch → explicit git fetch fallback guidance. Safe ordering.
  3. Walk stops at merge commits with a documented rationale — correct for this repo's all-squash-merge master history.
  4. No injection surface: all git calls use list argv; no shell interpolation; no config mutation.
  5. Doc-count guard consistent: Agent.md 1082 → 1087 matches the 5 new tests exactly.
  6. CI green: test + test-windows both pass (run 32876776090); MERGEABLE/CLEAN.

No issues found.

@argszeroargszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260826-013702 (independent re-review)

Head unchanged since cycle cyc20260826-013007 review (0484090). Fresh verification this cycle: checked out the PR head in a clean worktree and ran the hermetic suite — tests/test_sync_master_from_api.py = 5/5 passed (byte-exact sha reproduction for unsigned + GPG-signed commits, mismatch raises). CI green (test + test-windows, run 32876776090); MERGEABLE/CLEAN. No issues found. 2/3.

@argszeroargszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260826-014334 (3rd independent vote)

Head unchanged (0484090); CI green (test + test-windows, run 32876776090); MERGEABLE/CLEAN. Prior reviews from cycles cyc20260826-013007 + cyc20260826-013702 (incl. local hermetic 5/5 pass at PR head) remain valid. Three consecutive LGTMs from distinct cycles, no ❌ in between — mergeable.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@pm25coder@argszero