Uh oh!
There was an error while loading. Please reload this page.
refactor: singular + enforced toolkit data contract - #1105
Merged
Conversation
Deletion and documentation accuracy only; no behavior change. - CLAUDE.md documented `pnpm build` as a three-stage pipeline ending in pagefind. Pagefind does not exist in this repo (search is an external Algolia crawler); the build is a single `next build --webpack`. Adds a Vale install note, since `pnpm vale:check` is documented as required but vale is a Go binary with no npm dependency. - .gitignore reserved `public/toolkit-markdown/` for a build step that no longer exists, and both .gitignore and the Makefile referenced `make_toolkit_docs/`, a Python directory that was removed. `make mcp-server-docs` was therefore a broken target. - `data/toolkits/jira.json` was an unreferenced 133 KB copy at the repo root; the live data is under `toolkit-docs-generator/data/toolkits/`. - Drops unused dependencies (zustand, turndown, @mdx-js/react) and redundant direct declarations that are supplied transitively (@theguild/remark-mermaid via nextra, baseline-browser-mapping via next, unist-util-visit-parents, mdast-util-to-string). Moves chalk to devDependencies and consolidates the two colour libraries onto it. - Adds @types/hast so neutralize-emails.tsx can use unist-util-visit instead of a hand-rolled tree walk. - The nightly generator workflow ran `pnpm build` with a working-directory that has no package.json, so pnpm resolved upward and executed the root Next production build. The step that follows runs the CLI through tsx and needs no build. - Renames ignored-toolkits.txt/excluded-toolkits.txt to skip-toolkits.txt/remove-toolkits.txt, which say what they do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The header comment still listed "Build the toolkit docs generator" as step 1 after that step was removed. Renumber the remaining three. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The latest updates on your projects. Learn more about Vercel for GitHub.
|
teallarson
commented
Jul 31, 2026
ContributorAuthor
@cursor review |
Two isolated behavior fixes. Static rendering: the root layout awaited headers() to read "x-pathname" and derive a locale. Awaiting headers() in the root layout opts the entire route tree out of static rendering, so every page — including all 117 toolkit pages, which are pure functions of committed JSON — was server-rendered on demand. The derived locale was always "en": proxy.ts redirects every non-English locale to /en and getPreferredLocale returns "en" unconditionally. The site paid full dynamic rendering to compute a constant. This is not an i18n change. TranslationBanner and the dictionary plumbing stay in place; restoring real i18n means an app/[lang]/ route segment, which is the correct Next pattern regardless. Sitemap: app/sitemap.ts skips any directory whose name contains "[", which is right for directory walking but meant all 117 toolkit pages were absent from sitemap.xml — the largest content section on the site. Merges in listValidIntegrationLinks() from app/_lib/toolkit-static-params.ts, the same enumeration the integrations index uses, and dedupes against the authored partner pages the disk walk already finds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eliminates the duplication that lets these subsystems drift.
Redirects were a 909-line array inside next.config.ts, so three consumers
regex-parsed the config as text — one of them carrying a second "reversed"
regex that existed only because a human might write {destination, source}
instead of {source, destination}, a problem that only exists when you parse
text instead of importing data. The array now lives in a typed redirects.ts
and every consumer imports it. Two further consumers that also parsed the
config as text (scripts/update-internal-links.ts and
tests/integration-index-links.test.ts) would have silently found zero
redirects, so they move to the import too. --auto-fix now appends to the
data file, and its six scattered "Auto-added redirects" marker blocks
collapse to one append point. The resolved array is byte-identical: 156
entries, same order.
check-redirects-utils.ts had 425 lines of tests but was imported by nothing
except its own test file, while the code that actually runs — the
pre-commit hook's scripts/check-redirects.ts — kept private copies of the
same six helpers. The tests guarded a copy while the shipping
implementation was untested. The module moves to scripts/lib/ and the
shipping script now imports it.
Toolkit primitives (data dir, toKebabCase, normalizeToolkitId, the category
list, the *Api heuristic, docsLink→slug) existed in 2-7 copies, one pair
carrying a "must stay in sync" comment. They collapse into
toolkit-docs-generator/src/shared/, which both halves can import. All seven
data-dir consumers now honor TOOLKIT_DATA_DIR; previously only two did.
The Node-only path resolution lives in its own module because client
components reach the primitives through the integrations index, and a
node:* import anywhere in that graph fails the webpack browser build.
The consumer-side contract was a four-field duck-check that never verified
tools was an array, followed by an unchecked cast — while toToolkitSummary
immediately calls data.tools.map(). The generator's Zod schemas are now the
single shared contract, the 522-line hand-written mirror is z.infer, and
zod moves to dependencies because it enters the app's runtime path.
Corruption is now loud and absence stays quiet: three catch blocks treated
missing, unparseable, and schema-invalid identically, so a malformed file
from the nightly PR silently dropped a toolkit and 404'd. Unparseable or
invalid now throws with the file path and the Zod issues, failing the
build. An unrecognized category throws instead of being coerced to
"others", which had no route directory and would have made every toolkit
in a new category a clickable card pointing at a 404.
One cache()-wrapped loader replaces 11 full passes over the data directory
per build, and removes the scan-every-file miss path that a burst of
unknown IDs could otherwise trigger at request time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>teallarsonforce-pushed
the
chore/single-source-of-truth
branch
from
August 3, 2026 19:39
072b21b to
35ae4f2Compareteallarson
marked this pull request as ready for review
August 4, 2026 16:04
Uh oh!
There was an error while loading. Please reload this page.
Deduplicate production readToolkitData calls so generateMetadata and Page share one in-flight read, recover failed cache entries, and drop stale "others" references from broken-link-check and sidebar sync docs. Co-authored-by: Cursor <cursoragent@cursor.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Deriving DEFAULT_TOOLKIT_DATA_DIR from import.meta.url broke /api/toolkit-data/[toolkitId] on deploy. Webpack replaces import.meta.url with a compile-time constant — the source file's absolute path on the build machine — so the resolved directory is /vercel/path0/... and does not exist inside the deployed function. Build-time static generation still succeeded, which is why nothing failed locally or in CI; the preview returned 500 for every toolkit data request while production returned 200. Anchoring on process.cwd() restores the path production already uses, and adds a test that chdirs before importing the module so a self-relative default fails loudly instead of only on deploy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The jq program is single-quoted, so the shell passed the backslashes through untouched and jq read `\\n` as an escaped backslash plus "n". Slack rendered the whole alert as one line with literal `\n` in it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
app/sitemap.ts hand-rolled the same TOOLKIT_DATA_DIR expression the shared resolver already provides, so the two could drift. The README pointed at .github/scripts/sync-toolkit-sidebar.ts, which lives in toolkit-docs-generator/scripts/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit bfdd661. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
readToolkitDataUncached resolves a lookup from two forms of its input: the normalized id (direct file, byNormalizedId) and the lowercased raw string (bySlug). Keying the cache on only the normalized form collapsed variants that resolve differently, and NotionToolkit is a real instance in the generated data — its file is notiontoolkit.json and "notion" reaches it only through the slug map. So on a warm instance the first variant requested decided the answer for all of them. /api/toolkit-data/no-tion served the full Notion payload where production 404s, and in the other order a miss pinned null so the real Notion page lost its tool detail and markdown export. The route takes arbitrary ids, so any request could trigger either direction. Key on the lowercased id instead, and retain only lookups that found data: concurrent callers still share the in-flight promise, but a resolved miss is dropped so an absent toolkit resolves once generated and arbitrary ids can't grow the map without bound. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
teallarson added a commit
that referenced
this pull request
Aug 5, 2026
main squash-merged #1105 as ab9a282, and this branch predated the last five commits of that PR, so the merge was a real three-way conflict across 12 files rather than squash noise. Resolution rule: main wins on the data-contract and runtime fixes it added; S2 wins on the check enablement it exists to deliver. - main wins wholesale on the 5 files S2 never touched: the toolkit data cache keying, the cwd-anchored data-dir resolver, the sitemap resolver share, the Slack alert newlines, and the page-factory comment. - S2 wins on the .js import specifiers in scripts/. The generator project is module: NodeNext with declaration + outDir, so it emits and .ts specifiers are illegal there. main never caught this because its tsconfig only included src/**. - S2 wins on docsLink ?? null: ToolkitSlugSource declares docsLink?: string | null and the project sets exactOptionalPropertyTypes, so passing undefined is an error. Behaviour is unchanged. - sync-toolkit-sidebar.test.ts took both sides: S2's required navGroup field plus main's corrected categories, since the removed "others" catch-all no longer exists. Root and generator typecheck clean, 801 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

This PR makes redirects and toolkit data use shared, typed sources instead of duplicated definitions and text parsing.
Before → after
next.config.ts; consumers parsed the file as text.redirects.ts; consumers import the data directly. The 156-entry array is unchanged.TOOLKIT_DATA_DIRconsistently.ToolkitData.null; invalid JSON or schema data throws with the file path and validation error.othersroute.othersis removed.Compatibility
Verification
pnpm testpasses.pnpm exec tsc --noEmitpasses.pnpm lintreports no errors and the same three pre-existing complexity warnings.One intentional trade-off: a corrupt toolkit file now fails the directory load instead of affecting only that toolkit. In production, the build catches this before requests are served.
Note
Medium Risk
Changes affect static route generation, sitemap/redirect correctness, and production toolkit API reads—invalid JSON now fails builds/requests instead of hiding toolkits, which is safer but stricter.
Overview
Redirects move out of
next.config.tsinto importableredirects.ts(same rules, no behavior change). CI, pre-commit,check-redirects,update-links, and sitemap tests now use that module instead of regex-parsing config.Toolkit docs data is a single contract: UI types are
z.inferfrom generatortoolkit-schemas;readToolkitFile/readToolkitIndexvalidate with Zod. Missing files still returnnull; bad JSON or schema mismatches throw with paths so builds break on corrupt nightly output.loadAllToolkitData+ production lookup caching dedupe reads; slug vs normalized id cache keys stay distinct for the API route.Routing / integrations: slug, category, and data-dir helpers live in
toolkit-docs-generator/src/shared/; the fakeotherscatch-all is gone—no category meanstoIntegrationLinkreturnsnulland cards are non-clickable; unknown categories throw. Sidebar sync and filters follow the same category list (design-systemCATEGORIESfor filters).DX / ops: webpack
extensionAliasfor shared.jsimports;zodis a runtime dependency; toolkit docs CI gets a Slack alert job on generation failure; new tests cover data dir, cache, parity, and category route dirs.Reviewed by Cursor Bugbot for commit 53e7df1. Bugbot is set up for automated code reviews on this repo. Configure here.