Skip to content

ci: retire the Codecov upload, re-aim the loud verdict at the coverage gate - #5769

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-5436-retire-codecov-upload
Aug 23, 2026
Merged

ci: retire the Codecov upload, re-aim the loud verdict at the coverage gate#5769
os-zhuang merged 3 commits into
mainfrom
claude/issue-5436-retire-codecov-upload

Conversation

@claude

@claudeclaudeBot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Fixes#5436

Maintainer ruling of 2026-08-22, Option B: retire the Codecov upload rather than set the secret. CODECOV_TOKEN was never set on this repository and Codecov no longer accepts tokenless uploads, so Upload coverage to Codecov failed server-side (Token length: 0, Token required - not valid tokenless upload) on every push after #5403 landed. Option A was not taken; Option C (skip the upload when the secret is absent) stays rejected — it restores exactly the quiet green that #5403 removed.

What replaces the upload

The merged report is published as the coverage-report artifact, retention-days: 7 to match the shard blobs it derives from, if-no-files-found: error because "the merge wrote no report" is a finding rather than a shrug. The existing step summary is unchanged. actions/upload-artifact@v7 matches the five existing pins in this workflow.

The publish step deliberately does not inherit the implicit success(). It runs under !cancelled() && needs.test-coverage.result == 'success' && steps.merge.outcome != 'skipped', because when the merge fails on the thresholds the report exists and is precisely the artifact someone wants to open. The job still goes red — the verdict step requires merge=success.

What the loud step now asserts, and what makes it red

#5403 built a verdict step whose purpose was to report shards=… upload=… and go red when Codecov received nothing. With no upload, its UPLOAD: half has no subject. The step is re-aimed rather than deleted, onto the claim that survives:

the job is green only when all four shards passed, all four blob reports arrived, they merged into one report, the configured thresholds were enforced over that whole merged map, and the report was published as a retrievable artifact — and red, with an error annotation naming which of those did not happen, every other time.

It reports shards=$SHARDS blobs=$BLOBS merge=$MERGE report=$REPORT and still runs under if: always(), so the path where every step above was skipped is red rather than green. This is a stronger claim than the one it replaces: the old red said "the dashboard is stale", the new red says "the coverage gate did not run on this commit" — and since the shard legs override the thresholds to zero, this job is the only place that gate can run at all.

Two reds, deliberately distinguished, because they call for opposite actions:

conditionannotationwhat it means
merge=failureandreport=successCoverage thresholds breacheda real coverage regression to fix
anything else short of full successCoverage gate did NOT runa broken lane to repair

Keying the first on report=success is load-bearing, not decorative. generateReports() writes the configured reporters in a loop and only then evaluates the thresholds, so a breach leaves a complete coverage/ behind and the publish step succeeds; a merge that dies earlier (unreadable blob, version mismatch) writes no report, if-no-files-found: error makes the publish step red too, and the run falls through to the second row — where it belongs. Without this key, a broken merge would be annotated as a coverage regression.

Invariants preserved

  • The gate. Thresholds are enforced by the merge step over the whole merged map. It was never Codecov's job and it is untouched here. The shard legs still override the thresholds to zero (ci.yml lines 556-557), which is what makes the merge step the sole enforcement point.
  • The four-shard refusal. "A wrong coverage number is worse than a missing one" is preserved verbatim; only the word upload in its message became merge.
  • The job nameTest (coverage) is unchanged — it is a named context that scripts/dependabot-merge-gate.mjs reads.

Demonstrated, not asserted

The acceptance asked for the threshold failure to be shown. Against one real blob set reporting 38.4% lines, only the threshold moved between the two runs:

legflagexitcoverage-final.json
A--coverage.thresholds.lines=380written, 201959 bytes
B--coverage.thresholds.lines=391ERROR: Coverage for lines (38.4%) does not meet global threshold (39%)written, 201959 bytes

Leg B also establishes the byte-identical report survives a breached gate, which is what licenses the publish step's !cancelled(). #5403's CI ablation could not separate these legs — its blobs carried a failing shard, so both went red for that reason — which is why the demonstration was owed here.

Codecov sweep

Every remaining reference in the repository was enumerated, not sampled:

Changeset

node scripts/check-changeset-presence.mjs answers that none is owed: 0 of them under the src/ of a package the release covers. One is added anyway with empty frontmatter, which AGENTS.md §161 calls a first-class way to declare "publishes nothing" — the goal being to declare once, not to force a release. It matches how the two sibling CI-only changesets in .changeset/ are written.

Verification

Exit codes captured before any pipe; each line quotes the tool's own verdict.

commandresult
vitest run scripts/__tests__/61 files, 1631 tests passed, exit 0
vitest run scripts/__tests__/ci-cd-pipeline-doc.test.ts32 passed, exit 0 (the docs-page pin)
vitest run scripts/__tests__/dependabot-merge-gate.test.ts35 passed, exit 0
node scripts/check-changeset-presence.mjsexit 0, No source of a released package changed in this range
node scripts/check-changeset-no-major.mjsexit 0, No changeset declares a major bump
python3 yaml.safe_load on ci.ymlparses; job name Test (coverage) preserved, no Codecov step remains
grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' over all edited filesclean, no control bytes

Run against bc21c704b; main has since advanced to 649f4a541 (#5767), which touches useConsoleActionRuntime.tsx and its changeset only — no overlap with any file here.

⚠️ Maintainer follow-up (not actionable by an agent)

The CODECOV_TOKEN repository secret is now read by nothing in this repository. Deleting it from GitHub settings is a maintainer action and is deliberately not attempted here. It is inert either way — this is hygiene, not a blocker.


Generated by Claude Code

…e gate (#5436)
`CODECOV_TOKEN` was never set on this repository and Codecov no longer accepts
tokenless uploads, so `Upload coverage to Codecov` failed server-side on every
push after #5403 landed, leaving `Test (coverage)` permanently red. Per the
maintainer ruling of 2026-08-22 (Option B), retire the upload rather than set
the secret.
The merged report is now published as the `coverage-report` artifact (7 days,
matching the shard blobs it derives from) alongside the existing step summary.
#5403's loud verdict step is carried forward, not deleted. Its `UPLOAD:` half
lost its subject, so the step is re-aimed at the claim that survives: the job
is green only when all four shards passed, all four blobs arrived, they merged,
the thresholds were enforced over the whole merged map, and the report was
published. A threshold breach is annotated separately from a lane that never
delivered, keyed on `report=success`, because the two call for opposite
actions.
The gate is untouched and was never Codecov's: thresholds are enforced by the
merge step, and the shard legs still override them to zero. The refusal to
merge fewer than four shards is preserved verbatim.
Demonstrated, not asserted: against one real blob set reporting 38.4% lines,
`--coverage.thresholds.lines=38` exits 0 and `=39` exits 1 with
`ERROR: Coverage for lines (38.4%) does not meet global threshold (39%)`, with
a byte-identical `coverage-final.json` left behind on both legs.
Fixes#5436
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124Qg8rLvpXnQDwCmpKUmaJ
@os-zhuangClaude

Copy link
Copy Markdown
Contributor

Build Docs is red here — and it is red on the base branch too, so it is not this PR's

PM, domain:devx @ objectui seat (#5748). Stating this once, per the drive-to-green rule for a failure that is not the PR's: ⛔ no change is being pushed for it, and it is not being re-run.

Measured on both sides, not inferred:

whereresult
this PR, head faf951150Build Docsfailure — job 97152785515
main at bc21c704b, push run 12000 (32621190400)failed jobs: Build Docs and Test (coverage)

The failure chain in this PR's log is character-for-character the one objectui#5668 tracks:

./node_modules/.pnpm/pg-connection-string@2.14.0/…/esm/index.mjs [Client Component SSR]
./node_modules/.pnpm/@objectstack+spec@17.1.0_…/dist/index.mjs [Client Component SSR]
./node_modules/.pnpm/@objectstack+formula@17.1.0_…/dist/index.mjs [Client Component SSR]
./packages/core/dist/evaluator/fieldRules.js [Client Component SSR]
…
Failed: @object-ui/site#build

fs reaching the browser bundle via @objectstack/specpg-connection-string. This PR's diff is four files — .github/workflows/ci.yml, content/docs/guide/ci-cd-pipeline.md, scripts/dependabot-merge-gate.mjs, and a changeset — and touches nothing on that path. #5668 is open, pm:blocked, and owned by os-sales.

⚠️A correction to my own earlier reading, since it is on the record. The devx@objectui seat post (#5748 §2) says evidence was attached to #5668 in R1 showing Build Docspassed on a current-main-plus-no-op head. That reading was taken at ~04:39Z and it does not hold now: at bc21c704b (05:46Z) the job is failing on main. So #5668 is live again, or was never fully fixed — either way, "it passed on main earlier today" is not usable as a counter-probe against this PR and should not be quoted as one. The seat post is being corrected.

The other red on that same main run is Test (coverage) — which is the exact defect this PR fixes (#5436). That is a useful confirmation rather than a coincidence: the lane this PR repairs is currently red on the branch it will merge into.

Disposition: this PR is not proceeding to the queue while Build Docs is red, and it will take the base branch in and re-run once #5668 recovers. Every other check is being read individually by name as it completes; nothing here is being judged on the aggregate or the required set.


Generated by Claude Code

@os-zhuang
os-zhuang marked this pull request as ready for review August 23, 2026 10:11
@os-zhuang
os-zhuang added this pull request to the merge queueAug 23, 2026
Merged via the queue into main with commit d600ee2Aug 23, 2026
22 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5436-retire-codecov-upload branch August 23, 2026 10:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test (coverage) is red on every push to main because CODECOV_TOKEN is unset — the four shards are green, the upload is rejected

2 participants

@os-zhuang@claude