feat(ui): add llms.txt generation for npm packages by lukeocodes · Pull Request #1382 · npmx-dev/npmx.dev

lukeocodes · 2026-02-11T13:43:17Z

Summary

Adds llms.txt and llms_full.txt support across all package URL patterns.

Supported `llms.txt` routes

Route	Example	Preview
Root discovery	`/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/llms.txt
Unscoped package	`/package/nuxt/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/nuxt/llms.txt
Unscoped versioned	`/package/nuxt/v/3.15.0/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/nuxt/v/3.15.0/llms.txt
Scoped package	`/package/@deepgram/sdk/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/@deepgram/sdk/llms.txt
Scoped versioned	`/package/@deepgram/sdk/v/4.0.0/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/@deepgram/sdk/v/4.0.0/llms.txt
Org listing	`/package/@deepgram/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/@deepgram/llms.txt
Full (with agent files)	`/package/nuxt/llms_full.txt`	https://npmx-9wsqop19f-npmx.vercel.app/package/nuxt/llms_full.txt
Shorthand redirect	`/nuxt/llms.txt` → `/package/nuxt/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/nuxt/llms.txt
Shorthand versioned	`/nuxt/v/3.15.0/llms.txt` → `/package/nuxt/v/3.15.0/llms.txt`	https://npmx-9wsqop19f-npmx.vercel.app/nuxt/v/3.15.0/llms.txt

llms.txt — README + metadata only
llms_full.txt — README + metadata + agent instruction files (CLAUDE.md, .cursorrules, etc.)

Supported `.md` routes

Route	Example	Preview
Unscoped package	`/package/nuxt.md`	https://npmx-9wsqop19f-npmx.vercel.app/package/nuxt.md
Scoped package	`/package/@deepgram/sdk.md`	https://npmx-9wsqop19f-npmx.vercel.app/package/@deepgram/sdk.md
Shorthand redirect	`/nuxt.md` → `/package/nuxt.md`	https://npmx-9wsqop19f-npmx.vercel.app/nuxt.md

.md — README only

Test plan

Root /llms.txt returns 200 with text/markdown content type
Unscoped and scoped package routes return 200
Versioned package routes return 200 (not 404)
Org listing route returns package list with full llms.txt URLs
llms_full.txt includes agent instruction files, llms.txt does not
Shorthand URLs (e.g. /nuxt/llms.txt) redirect 301 to canonical paths
Existing package page routes are unaffected
Tests pass

vercel · 2026-02-11T13:43:23Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
npmx.dev	Ready	Preview, Comment	Mar 5, 2026 1:01pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
docs.npmx.dev	Ignored	Preview	Mar 5, 2026 1:01pm
npmx-lunaria	Ignored		Mar 5, 2026 1:01pm

codecov · 2026-02-11T13:45:22Z

Codecov Report

❌ Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
app/pages/package/[[org]]/[name].vue	0.00%	11 Missing and 2 partials ⚠️
app/pages/index.vue	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

coderabbitai · 2026-02-11T13:47:14Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds end-to-end LLM documentation support: new shared types (AgentFile, LlmsTxtResult) and a re-export in shared/types/index.ts; server utilities to discover and fetch agent files, assemble llms.txt/llms_full.txt and package README content; a new middleware (server/middleware/llm-docs.ts) to serve package, org and root llms routes and raw README .md; vitest test alias for #server; comprehensive unit tests for the llm-docs utilities; and a smoke-test script for the new routes. Also updates canonical redirect middleware to preserve optional suffixes.

Possibly related issues

feat: generate llms.txt for packages from agent instruction files #1378 — The PR implements per-package and org-level llms.txt generation and serving, which matches the feature described in this issue.

Suggested reviewers

danielroe

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description check	✅ Passed	The pull request description comprehensively details the llms.txt and llms_full.txt feature addition with supported routes, examples, test plan, and implementation notes.
Merge Conflict Detection	✅ Passed	✅ No merge conflicts detected when merging into `main`

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

coderabbitai

Actionable comments posted: 1

lukeocodes · 2026-02-13T00:53:59Z

@danielroe when merging please add @BYK as a co-author from #151

coderabbitai

Actionable comments posted: 3

🧹 Nitpick comments (1)

server/middleware/llm-docs.ts (1)
35-135: Split the middleware into smaller helpers.
This handler covers multiple routes and behaviours in one block and is well over the “~50 lines” guideline. Consider extracting .md handling, root handling, org handling, and package parsing into dedicated functions.
As per coding guidelines: Keep functions focused and manageable (generally under 50 lines).

Add AgentFile and LlmsTxtResult interfaces for llms.txt generation and export from the shared types barrel.

Add discoverAgentFiles, fetchAgentFiles, generateLlmsTxt, and handleLlmsTxt orchestrator for llms.txt generation from npm packages.

Serve llms.txt at /api/registry/llms-txt/[...pkg] following existing registry API patterns with cached event handler and SWR.

Cover discoverAgentFiles, fetchAgentFiles, and generateLlmsTxt including root files, directory scanning, graceful failures, scoped packages, and full/minimal output generation.

Use Nitro server routes at /package/.../llms.txt instead of an API route with middleware rewriting. Single handler re-exported across four route files for unscoped, scoped, and versioned URL patterns.

…txt content Add createPackageLlmsTxtHandler factory for DRY route creation. handleLlmsTxt now accepts includeAgentFiles option to control whether agent instruction files are included (llms_full.txt) or omitted (llms.txt). Add handleOrgLlmsTxt for org-level package listings and generateRootLlmsTxt for the root /llms.txt discovery page. Simplify route handlers to single-line factory calls.

Add server middleware to handle llms.txt routes that Nitro's radix3 file-based router cannot resolve (parameterized intermediate segments don't match literal children). Handles versioned package paths, org-level package listings, and root /llms.txt discovery page. Remove broken versioned route files and add llms_full.txt routes.

Extend canonical redirect regexes with optional /llms.txt and /llms_full.txt suffix capture groups so shorthand URLs like /nuxt/llms.txt redirect to /package/nuxt/llms.txt. Add explicit /llms.txt root path skip to prevent it matching as a package name.

Add ISR rules for llms_full.txt and root /llms.txt routes in nuxt.config.ts. Add #server alias to vitest config for resolving server utility imports in unit tests.

lukeocodes · 2026-03-05T09:06:04Z

@danielroe @patak-cat ticky ticky bang bang? <3

Add <link rel="alternate"> tags to HTML head for LLM-friendly document discovery on homepage, org pages, and package pages.

shi-gg · 2026-03-05T15:11:52Z

From what I can see, there is no llms.txt for a packages API docs, right?

i.e.: https://npmx.dev/package-docs/linkdave/v/0.1.2

lukeocodes · 2026-03-05T15:15:56Z

From what I can see, there is no llms.txt for a packages API docs, right?
i.e.: https://npmx.dev/package-docs/linkdave/v/0.1.2

Nope, that seems out of scope for a initial version though

shi-gg · 2026-03-05T15:17:08Z

fair enough — though I would love to see that in the future

lukeocodes · 2026-03-05T15:45:58Z

fair enough — though I would love to see that in the future

yeh me too, this felt a little invasive for me for a single PR so i thought i'd limit it much more than what i've done

shi-gg

lgtm, but

shi-gg · 2026-03-05T16:33:19Z

+ const full = path.endsWith('/llms-full.txt')
+ const suffix = full ? '/llms-full.txt' : '/llms.txt'
+
+ // Root /llms.txt


a thing that I personally dislike is the amount of comments that explain very simple statements.
If a comment is needed, the code should be made more readable.

shi-gg · 2026-03-05T16:50:56Z

I just noticed this happening on paths that I haven't opened before

Screencast.From.2026-03-05.17-49-51.webm

serhalp · 2026-03-16T23:09:48Z

@danielroe you're assigned to this one — is that stale? 🤔

serhalp · 2026-03-16T23:10:25Z

@lukeocodes are you still up for seeing this one through?

lukeocodes · 2026-03-17T12:13:36Z

@lukeocodes are you still up for seeing this one through?

yup i need to find time to do the follow ups. this is creeping a bit

gameroman

I think there is some overlap with #151 in terms of functionality

https://npmx-juq6izdv3-npmx.vercel.app/vite.md and https://npmxdev-git-fork-lukeocodes-feat-llms-txt-npmx.vercel.app/package/vite/llms.txt for example

lukeocodes · 2026-04-30T15:57:48Z

@danielroe you can close or find someone to pick this one up, it was a little too stop-start trying to get this out for me.

i think my feedback in the discord was that we're expecting these PRs to be perfect at the point of release, instead of introducing them as good and iterating

vercelBotdeployed to Preview – npmx.dev February 11, 2026 13:45 View deployment

coderabbitaiBot reviewed Feb 11, 2026
View reviewed changes

Comment threadserver/utils/llm-docs.ts
Comment threadserver/utils/llm-docs.ts

lukeocodes marked this pull request as draft February 11, 2026 13:51

vercelBotdeployed to Preview – npmx.dev February 11, 2026 15:28 View deployment

okineadev mentioned this pull request Feb 12, 2026
feat: add markdown output support for package pages #151
Closed

vercelBotdeployed to Preview – npmx.dev February 12, 2026 11:39 View deployment

vercelBotdeployed to Preview – npmx.dev February 12, 2026 13:32 View deployment

lukeocodes changed the title ~~feat(llms-txt): add llms.txt generation for npm packages~~feat(ui): add llms.txt generation for npm packagesFeb 12, 2026

vercelBotdeployed to Preview – npmx.dev February 12, 2026 14:10 View deployment

lukeocodes marked this pull request as ready for review February 12, 2026 14:13

vercelBotdeployed to Preview – docs.npmx.dev February 12, 2026 14:15 View deployment

vercelBotdeployed to Preview – npmx.dev February 12, 2026 14:15 View deployment

vercelBotdeployed to Preview – npmx.dev February 13, 2026 00:31 View deployment

coderabbitaiBot reviewed Feb 13, 2026
View reviewed changes

Comment threadserver/utils/llm-docs.ts

vercelBotdeployed to Preview – npmx.dev February 13, 2026 00:41 View deployment

vercelBotdeployed to Preview – npmx.dev February 13, 2026 00:54 View deployment

coderabbitaiBot reviewed Feb 13, 2026
View reviewed changes

Comment threadscripts/smoke-test-llm-docs.sh
Comment threadserver/middleware/canonical-redirects.global.ts
Comment threadserver/utils/llm-docs.ts

lukeocodes added 9 commits March 5, 2026 07:32

feat(types): add llms-txt type definitions
b92b7f1
Add AgentFile and LlmsTxtResult interfaces for llms.txt generation and export from the shared types barrel.

feat(llms-txt): add core utility functions
31aeabd
Add discoverAgentFiles, fetchAgentFiles, generateLlmsTxt, and handleLlmsTxt orchestrator for llms.txt generation from npm packages.

feat(llms-txt): add API route for llms.txt generation
557bccd
Serve llms.txt at /api/registry/llms-txt/[...pkg] following existing registry API patterns with cached event handler and SWR.

test(llms-txt): add unit tests for utility functions
634148c
Cover discoverAgentFiles, fetchAgentFiles, and generateLlmsTxt including root files, directory scanning, graceful failures, scoped packages, and full/minimal output generation.

feat(llms-txt): replace API route with file-based server routes
165a605
Use Nitro server routes at /package/.../llms.txt instead of an API route with middleware rewriting. Single handler re-exported across four route files for unscoped, scoped, and versioned URL patterns.

chore(llms-txt): add ISR cache rules and vitest server alias
b5a17c9
Add ISR rules for llms_full.txt and root /llms.txt routes in nuxt.config.ts. Add #server alias to vitest config for resolving server utility imports in unit tests.

feat(llm-docs): add alternate link tags for llms.txt and .md routes
16c42ce
Add <link rel="alternate"> tags to HTML head for LLM-friendly document discovery on homepage, org pages, and package pages.

vercelBotdeployed to Preview – npmx.dev March 5, 2026 12:18 View deployment

Merge branch 'main' into feat/llms-txt
b4e40e3

vercelBotdeployed to Preview – npmx.dev March 5, 2026 13:01 View deployment

shi-gg reviewed Mar 5, 2026
View reviewed changes

danielroe self-assigned this Mar 5, 2026

serhalp added the needs review This PR is waiting for a review from a maintainer label Mar 15, 2026

serhalp added the stale This has become stale and may be closed soon label Apr 10, 2026

serhalp added this to the Up Next milestone Apr 10, 2026

github-project-automationBot added this to Releases Apr 10, 2026

github-project-automationBot moved this to Backlog in Releases Apr 10, 2026

github-actionsBot removed the stale This has become stale and may be closed soon label Apr 18, 2026

serhalp unassigned danielroe Apr 20, 2026

gameroman reviewed Apr 30, 2026
View reviewed changes

github-actionsBot added stale This has become stale and may be closed soon and removed stale This has become stale and may be closed soon labels May 31, 2026

github-actionsBot added the stale This has become stale and may be closed soon label Jul 2, 2026

ghostdevv removed the stale This has become stale and may be closed soon label Jul 6, 2026

gameroman added the needs discussion An idea that needs more discussion to understand the scope and impact. label Aug 4, 2026

github-actionsBot added the stale This has become stale and may be closed soon label Sep 4, 2026

Uh oh!

Uh oh!

Conversation

lukeocodes commented Feb 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Supported llms.txt routes

Supported .md routes

Test plan

Uh oh!

vercelBot commented Feb 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecovBot commented Feb 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coderabbitaiBot commented Feb 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Possibly related issues

Suggested reviewers

Uh oh!

coderabbitaiBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coderabbitaiBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lukeocodes commented Feb 13, 2026

Uh oh!

coderabbitaiBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lukeocodes commented Mar 5, 2026

Uh oh!

shi-gg commented Mar 5, 2026

Uh oh!

lukeocodes commented Mar 5, 2026

Uh oh!

shi-gg commented Mar 5, 2026

Uh oh!

lukeocodes commented Mar 5, 2026

Uh oh!

shi-gg left a comment

Choose a reason for hiding this comment

Uh oh!

shi-ggMar 5, 2026

Choose a reason for hiding this comment

Uh oh!

shi-gg commented Mar 5, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhalp commented Mar 16, 2026

Uh oh!

serhalp commented Mar 16, 2026

Uh oh!

lukeocodes commented Mar 17, 2026

Uh oh!

gameroman left a comment

Choose a reason for hiding this comment

Uh oh!

lukeocodes commented Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

lukeocodes commented Feb 11, 2026•
edited
Loading

Supported `llms.txt` routes

Supported `.md` routes

vercelBot commented Feb 11, 2026•
edited
Loading

codecovBot commented Feb 11, 2026•
edited
Loading

coderabbitaiBot commented Feb 11, 2026•
edited
Loading

shi-gg commented Mar 5, 2026•
edited
Loading