Skip to content

docs site: any single-segment path containing a dot renders the homepage with 200 (soft-404 class) #12233

Description

@os-zhuang

One-liner

/foo.txt, /ads.txt, /sitemap_index.xml, /security.txt — every single-segment path with a dot in it returns 200 with the full homepage. Paths without a dot 404 correctly. So the site publishes an unbounded set of duplicate homepages at exactly the URLs crawlers probe by default.

Measured (local dev server, same routing in production)

/foo.txt 200 title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/ads.txt 200 title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/sitemap_index.xml 200 title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/security.txt 200 title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/this-page-does-not-exist 404 ← no dot, correct

Root cause

apps/docs/proxy.ts matcher excludes dotted paths from locale rewriting:

matcher: ['/((?!api|_next/static|_next/image|favicon.ico|.*\\..*).*)'],

so /robots.txt skips the proxy and falls straight through to the single-segment dynamic route app/[lang]/page.tsx, which matches with lang = "robots.txt" and renders HomePage() — it never checks whether lang is a real locale. A dotless path is rewritten to /en/<path>, becomes two segments, matches nothing, and 404s properly.

Expected

[lang] accepts only the locales in lib/i18n.ts. Either export const dynamicParams = false alongside the existing generateStaticParams, or an explicit if (!i18n.languages.includes(lang)) notFound() in app/[lang]/layout.tsx — whichever the Next 16 app router honours for both the static and the dev path. Verify both, do not assume.

Why it matters

Soft-404s at this scale burn crawl budget and split ranking signal across junk URLs; and the two paths that matter most (/robots.txt, /sitemap.xml) are in the affected set, so this card and the robots/sitemap card must both land for either to be worth anything.

Acceptance

  • /foo.txt, /ads.txt, /security.txt, /anything.html all return 404
  • /robots.txt and /sitemap.xml still resolve to their real routes (regression check against the sibling card)
  • /docs, /docs/<any real page>, /blog unchanged
  • a test or gate pins the behaviour so the catch-all cannot silently return

Source

Found in an SEO review of the docs site (apps/docs) run on 2026-08-25, measured against the local dev server and against production. The canonical origin is https://objectstack.ai — maintainer ruling recorded in #10659:

这个仓的文档站规范 URL 是 https://objectstack.ai

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions