Fix the nightly link check: tell a broken runner from a dead link, repair every dead link it found by shoumikhin · Pull Request #21694 · pytorch/executorch

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

shoumikhin · 2026-08-08T23:03:53Z

The nightly link-check / lint-urls job has been red for weeks. There are two separate problems behind that, and this PR fixes both.

Problem 1: the script cannot tell a dead host from a broken runner

scripts/lint_urls.sh gets 000 back from curl when there is no HTTP response at
all. That happens for two very different reasons: the host is dead, or the runner
has no working egress. The script treated both as WARN and moved on.

That is unsafe in both directions:

A genuinely dead host is silently tolerated. The 000 case is exactly the one we
most want to catch.
More importantly it also disables the pull request gate. .github/workflows/lint.yml
runs this same script in diff mode on every pull request, where the only URLs
checked are the ones that pull request adds. A new link with a typo in the host
name gives 000 and passes today. A runner with no egress makes every link WARN
and the whole job green. Both were reproduced locally.

The fix

On 000, probe one known-good URL, https://api.github.com, which the runner
already depends on:

probe also returns 000, so this runner has no egress: WARN, as before.
probe returns anything else, so egress works and the problem is the host: retry
the URL once with a 60 second timeout, then judge it on that result, which for a
dead host means FAIL.

One deliberate addition worth calling out: the retry with the longer timeout is
more than the minimum needed to close the hole. It is there because the same URLs
were sampled across six nights of nightly logs and a small number of them, one or
two per night out of about 2100, return 000 once and then answer normally. The
default timeout is 10 seconds. Without the retry, this change would turn those
into hard failures and the nightly would stay red for no good reason.

Also removed: the check-host.net fallback

When the first request failed, the script asked check-host.net whether the URL
was reachable from elsewhere and could then overwrite the verdict. That is
removed. Note this makes the script stricter, not more lenient: on a 404 or a 500
the fallback could turn a FAIL into an OK. Dropping it means a real 404 stays a
404, and it also removes a third-party service from the critical path of a lint
job.

Problem 2: eight actually dead links

With problem 1 fixed the nightly would still be red, because there really are dead
links in the tree. The most recent nightly on main
(31350839957)
reported 7 x FAIL 404 and 1 x FAIL 000. All eight are handled here:

file	link	what happened	fix
`setup.py`	pybind/cmake_example `blob/master/setup.py`	that project deleted its `setup.py`, so a `master` link cannot work	pin to the last revision that still had it
`.ci/docker/common/install_openssl.sh`	pytorch/pytorch `blob/main/.ci/docker/common/install_openssl.sh`	the file moved on `main`	pin to a revision where it exists
`.ci/scripts/test_llava.sh`	Wikimedia basketball photo	the photo was renamed on Wikimedia, not deleted	point at the same photo under its current name
`docs/source/kernel-library-custom-aten-kernel.md`	`pytorch.org/cppdocs/library.html#...`	page and anchor both gone	current cppdocs library index
`test/models/export_program.py`	`pytorch.org/cppdocs/notes/tensor_indexing.html`	gone, and it was only mentioned in a comment	drop the URL, keep the sentence
`docs/source/success-stories.md`	an App Store listing	app delisted	drop the link, keep the text
`README.md`	`github.com/pytorch/executorch/stargazers`	returns 404 to an unauthenticated bot, works fine in a browser	`@lint-ignore`
`backends/arm/scripts/toolchain_utils.sh`	`musl.cc` toolchain tarball	serves fine to developers, has never once answered from a GitHub hosted runner	`@lint-ignore`

Two more URLs are cleaned up along the way: a truncated Qualcomm SDK URL in
backends/qualcomm/scripts/download_qnn_sdk.py that was never a real link, and a
Qwen issue link in examples/models/llama/runner/generation.py pointing at the
old repository name.

On the Wikimedia one specifically: examples/models/llava/README.md already
embeds the renamed photo and documents the caption the model produces for it,
which matches the EXPECTED_PREFIX the script asserts. So this restores the
original input rather than substituting a different picture. Worth knowing, and
the reason this is low risk: no workflow runs .ci/scripts/test_llava.sh. Both
pull.yml and trunk.yml carry the comment "llava gives segfault so not
covering", so llava is not in CI at all right now.

How this was verified

A whole-tree scan is nightly-only, so a pull request run here would only check the
handful of URLs this PR adds. To get real numbers, _link_check.yml was dispatched
in whole-tree mode against this branch's contents
(31417702080):

	main, nightly	this branch, whole tree
URLs scanned	2148	2147
OK	2116	2116
WARN	24	31
FAIL	8	0

Reconciling the two totals, since they should not match exactly:

11 URLs leave the scan: 5 replaced by a working equivalent, 4 deleted outright,
and 2 now marked @lint-ignore.
6 join it: the 5 replacements, plus https://api.github.com, which is now
written in lint_urls.sh as the probe target and so gets scanned itself.
That is 2148 minus 5, which is 2143. The remaining 4 are URLs that landed on main
during the day, in examples/models/muse-glimmer/, and are unrelated to this
change. All 4 pass.

The 7 extra WARNs are the same thing that produces the existing 24: sites that
return 403 to an automated client. Between the two runs, cppreference.com,
stackoverflow.com and vulkan.org happened to answer 403 instead of 200.
Those are already treated as warnings and are not affected by this PR.

The 000 handling itself was also exercised directly, in a scratch repository:

no egress at all: WARN 000, exit 0, matching a broken runner.
egress working, host that does not exist: FAIL 000, exit 1.
OK 200, WARN 403 and FAIL 404 all still behave as before.
diff mode still only looks at added lines.

pytorch-bot · 2026-08-08T23:03:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21694

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 114 Pending

As of commit f4bd5e6 with merge base 9cd0c12 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-08T23:04:46Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…pair every dead link it found

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

CopilotAI lite review requested due to automatic review settings August 8, 2026 23:03

shoumikhin temporarily deployed to cadence August 8, 2026 23:03 — with GitHub Actions Inactive

CopilotAI reviewed Aug 8, 2026

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 8, 2026

shoumikhin force-pushed the shoumikhin/linkcheck-fewer-false-positives branch from dbb9910 to 091bebfCompare August 8, 2026 23:33

CopilotAI review requested due to automatic review settings August 8, 2026 23:33

CopilotAI reviewed Aug 8, 2026

shoumikhin temporarily deployed to cadence August 8, 2026 23:33 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence August 9, 2026 00:03 — with GitHub Actions Inactive

shoumikhin force-pushed the shoumikhin/linkcheck-fewer-false-positives branch from 091bebf to 8ca5333Compare August 9, 2026 04:39

CopilotAI review requested due to automatic review settings August 9, 2026 04:39

shoumikhin requested review from kirklandsign, larryliu0820 and mergennachin as code owners August 9, 2026 04:39

CopilotAI reviewed Aug 9, 2026

shoumikhin temporarily deployed to cadence August 9, 2026 04:39 — with GitHub Actions Inactive

shoumikhin changed the title ~~Make the nightly link check trustworthy: stop failing on no-response, fix the one real dead link~~Fix the nightly link check: stop failing on no-response, repair every dead link it foundAug 9, 2026

shoumikhin force-pushed the shoumikhin/linkcheck-fewer-false-positives branch from 8ca5333 to 6813118Compare August 9, 2026 04:51

shoumikhin requested a review from abhinaykukkadapu as a code owner August 9, 2026 04:51

CopilotAI review requested due to automatic review settings August 9, 2026 04:51

shoumikhin requested a review from psiddh as a code owner August 9, 2026 04:51

CopilotAI reviewed Aug 9, 2026

shoumikhin temporarily deployed to cadence August 9, 2026 04:51 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence August 9, 2026 05:16 — with GitHub Actions Inactive

CopilotAI review requested due to automatic review settings August 9, 2026 05:55

shoumikhin temporarily deployed to cadence August 9, 2026 05:55 — with GitHub Actions Inactive

CopilotAI reviewed Aug 9, 2026

shoumikhin temporarily deployed to cadence August 9, 2026 06:20 — with GitHub Actions Inactive

CopilotAI review requested due to automatic review settings August 10, 2026 05:51

shoumikhin force-pushed the shoumikhin/linkcheck-fewer-false-positives branch from 6c4e82c to 70e4370Compare August 10, 2026 05:51

CopilotAI reviewed Aug 10, 2026

shoumikhin temporarily deployed to cadence August 10, 2026 05:51 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence August 10, 2026 06:53 — with GitHub Actions Inactive

Fix the nightly link check: tell a broken runner from a dead link, re…
f4bd5e6
…pair every dead link it found

shoumikhin force-pushed the shoumikhin/linkcheck-fewer-false-positives branch from 70e4370 to f4bd5e6Compare August 10, 2026 18:07

shoumikhin requested a review from digantdesai as a code owner August 10, 2026 18:07

CopilotAI review requested due to automatic review settings August 10, 2026 18:07

CopilotAI reviewed Aug 10, 2026

shoumikhin temporarily deployed to cadence August 10, 2026 18:07 — with GitHub Actions Inactive

github-actionsBot added ciflow/trunk module: arm Issues related to arm backend labels Aug 10, 2026

shoumikhin changed the title ~~Fix the nightly link check: stop failing on no-response, repair every dead link it found~~Fix the nightly link check: tell a broken runner from a dead link, repair every dead link it foundAug 10, 2026

shoumikhin temporarily deployed to cadence August 10, 2026 18:34 — with GitHub Actions Inactive

JakeStevens approved these changes Aug 10, 2026
View reviewed changes

shoumikhin merged commit 52467d5 into mainAug 10, 2026
689 of 690 checks passed

shoumikhin deleted the shoumikhin/linkcheck-fewer-false-positives branch August 10, 2026 19:04

psiddh mentioned this pull request Aug 12, 2026
Lint only what a PR adds: diff the link checks from the merge base #21765
Merged