Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Enable ci for external collaborators - #325

Draft
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci
Draft

Enable ci for external collaborators#325
pranaygp wants to merge 2 commits into
mainfrom
pranaygp/external-collaborator-ci

Conversation

@pranaygp

@pranaygppranaygp commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

Add new /run-ci command to allow repo admins to trigger a CI run for external collaborator PRs

Leaving in draft just so we remove the claude generated md files before actually merging :)

@changeset-bot

changeset-botBot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8e0fe49

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Nov 13, 2025

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-nextjs-workflow-webpackReadyReadyPreviewCommentDec 2, 2025 6:48pm
example-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-astro-workflowErrorErrorDec 2, 2025 6:48pm
workbench-express-workflowErrorErrorDec 2, 2025 6:48pm
workbench-hono-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nitro-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-nuxt-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-sveltekit-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workbench-vite-workflowReadyReadyPreviewCommentDec 2, 2025 6:48pm
workflow-docsReadyReadyPreviewCommentDec 2, 2025 6:48pm

Comment thread.github/workflows/trigger-ci.yml Outdated
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
try {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we pull these out into script files, like here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please do this

Comment thread.github/workflows/trigger-ci.yml Outdated
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.createComment({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we need to post a comment on the PR. Maybe just no-op.

Comment thread.github/workflows/trigger-ci.yml Outdated
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

// Add remote for the external fork if it's from a fork

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would always be from a fork, no?

Comment thread.github/workflows/trigger-ci.yml Outdated
script: |
const prDetails = ${{ steps.pr-details.outputs.result }};
const timestamp = new Date().getTime();
const ciBranchName = `ci-test/${prDetails.number}-${timestamp}`;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp seems unnecessary. PR numbers are unique.

Unless we are planning on creating a new branch each time this workflow is run, but my thinking was that we'd have a 1:1 relationship, and the test PR would update if the branch already exists and /run-ci is invoked for future updates on a given PR. Maybe this is fine for now as :itg: though

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot implement this

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #493, to work on those changes. Once the pull request is ready, I'll request review from you.

CopilotAI commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

@pranaygp I've opened a new pull request, #494, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan
* Extract inline scripts to separate files
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
* Fix indentation in create-ci-pr.js
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pranaygp <1797812+pranaygp@users.noreply.github.com>
@github-actions

github-actionsBot commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.038s (-15.5% 🟢)1.010s (~)0.972s1.00x
💻 LocalNext.js (Turbopack)0.042s (+10.0% 🔺)1.018s (~)0.976s1.11x
💻 LocalNitro0.043s (-1.6%)1.007s (~)0.964s1.14x
🐘 PostgresNitro0.341s (+48.1% 🔺)1.013s (-1.1%)0.672s9.08x
🐘 PostgresNext.js (Turbopack)0.372s (-7.2% 🟢)1.023s (~)0.651s9.89x
🐘 PostgresExpress0.409s (+72.0% 🔺)1.015s (~)0.605s10.89x

workflow with 1 step

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Next.js (Turbopack)0.089s (+12.2% 🔺)1.012s (~)0.923s1.00x
💻 LocalExpress0.094s (-18.2% 🟢)1.009s (~)0.915s1.05x
💻 LocalNitro0.110s (-1.8%)1.006s (~)0.896s1.23x
🐘 PostgresNitro1.209s (+4.3%)2.011s (~)0.802s13.57x
🐘 PostgresNext.js (Turbopack)1.212s (+35.1% 🔺)1.917s (+57.8% 🔺)0.705s13.60x
🐘 PostgresExpress1.222s (+10.6% 🔺)2.011s (~)0.790s13.71x

workflow with 10 sequential steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Express0.590s (-24.2% 🟢)1.009s (~)0.420s1.00x
💻 LocalNext.js (Turbopack)0.636s (+25.3% 🔺)1.010s (~)0.374s1.08x
💻 LocalNitro0.770s (-2.2%)1.005s (~)0.236s1.31x
🐘 PostgresNext.js (Turbopack)4.871s (-6.0% 🟢)5.418s (-10.0% 🟢)0.547s8.26x
🐘 PostgresExpress10.113s (-3.6%)11.024s (~)0.912s17.15x
🐘 PostgresNitro10.192s (+97.4% 🔺)11.024s (+89.6% 🔺)0.832s17.29x

workflow with 10 parallel steps

WorldFrameworkWorkflow TimeWall TimeOverheadvs Fastest
💻 Local🥇 Nitro0.388s (-3.3%)1.005s (~)0.618s1.00x
💻 LocalNext.js (Turbopack)0.391s (-4.0%)1.009s (~)0.618s1.01x
💻 LocalExpress0.415s (+4.4%)1.008s (~)0.593s1.07x
🐘 PostgresNext.js (Turbopack)1.183s (+47.2% 🔺)1.851s (+82.4% 🔺)0.668s3.05x
🐘 PostgresExpress1.640s (+3.1%)2.015s (~)0.375s4.23x
🐘 PostgresNitro1.697s (+53.8% 🔺)2.015s (+24.5% 🔺)0.318s4.38x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

WorldFrameworkWorkflow TimeTTFBWall TimeOverheadvs Fastest
💻 Local🥇 Express0.136s (-25.9% 🟢)1.000s (+0.8%)1.011s (~)0.875s1.00x
💻 LocalNext.js (Turbopack)0.144s (+26.4% 🔺)1.003s (~)1.014s (~)0.870s1.06x
💻 LocalNitro0.177s (-1.2%)0.992s (~)1.008s (~)0.831s1.30x
🐘 PostgresNext.js (Turbopack)0.768s (-46.1% 🟢)0.832s (-48.5% 🟢)1.016s (-49.6% 🟢)0.248s5.66x
🐘 PostgresExpress1.062s (-52.6% 🟢)0.982s (-65.0% 🟢)1.310s (-56.5% 🟢)0.248s7.82x
🐘 PostgresNitro2.372s (+212.3% 🔺)2.668s (+213.6% 🔺)3.014s (+198.6% 🔺)0.642s17.46x

Summary: Fastest Framework by World

Winner determined by most benchmark wins

World🥇 Fastest FrameworkWins
💻 LocalExpress3/5
🐘 PostgresNext.js (Turbopack)3/5

Summary: Fastest World by Framework

Winner determined by most benchmark wins

Framework🥇 Fastest WorldWins
Express💻 Local5/5
Next.js (Turbopack)💻 Local5/5
Nitro💻 Local5/5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world

⚠️Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,
head_repo_full_name: pr.data.head.repo.full_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with a TypeError when a PR is from a deleted fork, because it tries to access full_name on a null object. When a fork is deleted, GitHub returns head.repo as null in the PR data.

View Details
📝 Patch Details
diff --git a/.github/scripts/get-pr-details.js b/.github/scripts/get-pr-details.js
index 88a0ce5..3aad14c 100644
--- a/.github/scripts/get-pr-details.js+++ b/.github/scripts/get-pr-details.js@@ -5,6 +5,16 @@ module.exports = async ({ github, context }) => {
pull_number: context.issue.number
});
+ if (!pr.data.head.repo) {+ await github.rest.issues.createComment({+ owner: context.repo.owner,+ repo: context.repo.repo,+ issue_number: context.issue.number,+ body: '❌ Cannot run CI: The source fork for this PR has been deleted. The fork must exist in order to fetch the changes and run CI tests.'+ });+ throw new Error('Source fork has been deleted');+ }+
return {
head_ref: pr.data.head.ref,
head_sha: pr.data.head.sha,

Analysis

Unhandled null reference crash when PR is from deleted fork

What fails: The script .github/scripts/get-pr-details.js at line 11 crashes with TypeError: Cannot read properties of null (reading 'full_name') when attempting to access pr.data.head.repo.full_name without checking if head.repo is null.

How to reproduce:

  1. Create a pull request from an external fork
  2. Delete the external fork
  3. Comment /run-ci on the original PR
  4. The workflow "Trigger CI for External PRs" will crash in the "Get PR details" step

Result: Crashes with: Error: Cannot read properties of null (reading 'full_name')

Expected behavior: Should gracefully handle the deleted fork scenario and post a helpful error message to the PR, as documented in the octokit/rest.js issue #31. When a PR originates from an "unknown repository" (i.e., deleted fork), the GitHub API returns head.repo as null.

Fix implemented:

  • Added null check for pr.data.head.repo before accessing its properties
  • Posts a user-friendly error comment to the PR explaining why CI cannot run
  • Throws an error to fail the workflow gracefully instead of crashing

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate