Skip to content

docs: delete the phantom auth-failure alert and its two signal listings - #98

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-87-phantom-auth-metric
Aug 18, 2026
Merged

docs: delete the phantom auth-failure alert and its two signal listings#98
os-zhuang merged 1 commit into
mainfrom
claude/issue-87-phantom-auth-metric

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes#87

auth_failures_total is emitted by nothing, so ObjectOSAuthFailureSpike is an alert that stays green forever. It is deleted rather than reworded, and the two other places that recommended the same signal are corrected with it.

Premise re-verified

Independently re-measured on objectstack@origin/main at b057e53f4, with the reverse control a zero-hit needs:

git grep auth_failures_total origin/main → 0 files
git grep -l SEMCONV origin/main → 15 files ← control: grep and tree are fine

packages/observability/src/semconv.ts declares 13 names across HTTP, storage, cache, jobs and the package registry. No auth metric among them. The card is right.

The third option was measured, then rejected

#87 offered a cheap alternative to delete-or-wait: express the alert against http_requests_total{status=~"40[13]"}. Its own caveat was the part to test, so I read how the runtime actually answers before adopting it. It does not honestly cover what the alert claimed — three independent reasons, any one of which is disqualifying:

  1. The sign-in endpoints do not feed that counter at all.http_requests_total has exactly one emitter, instrumentRouteHandler (packages/runtime/src/observability/instrument.ts:106). It is applied by a Proxy that dispatcher-plugin.ts:700-721 builds over a localserver binding from ctx.getService('http.server'), never registered back as the service. plugin-auth mounts its routes on the raw Hono app — getRawApp(), then rawApp.all(basePath + '/*') (auth-plugin.ts:1622,2240). A failed password on /api/v1/auth/sign-in/email increments nothing.
  2. A rejected API key is indistinguishable from no credential.core/src/security/api-key.ts validates fail-closed by design — "anything ambiguous (missing, revoked, expired, malformed) resolves to no principal" — so a presented-and-rejected key and a request that carried nothing both arrive anonymous and both answer 401. This is exactly the case the card flagged as uncertain.
  3. 401/403 are dominated by traffic that is not an auth failure. 403 is this runtime's authorization code — PERMISSION_DENIED, export-permission denials, share-link denials, NOT_OVERRIDABLE, docs permission sets, the ADR-0069 auth gate (password expiry / enforced MFA) — all raised against users who authenticated perfectly well. 401 is every ordinary anonymous request.

So deleting is the better answer, which is the outcome the dispatch named as a good one.

What the pages say now, and what that does and does not cover

No alert is kept, so there is no YAML coverage claim to make. The replacement is prose, and here is precisely what it asserts:

  • It does cover telling an operator that no auth-failure series exists, so they stop looking for one, and that 401/403 counting is not a substitute — with the reason, so the next reader does not re-derive the idea and re-add the alert.
  • It does not cover giving them an auth-failure signal. There is none. The pointer to the Console's Auth audit view is deliberately scoped to sign-in activity, not to failures: plugin-audit's AuthSessionAuditEvent is a closed login | logout union with a required real sys_user id, so the ledger records successful sessions only. The page does not claim otherwise, and operate/audit-logs.mdx already documents that view.

Changes

FileChange
content/docs/operate/observability.mdxObjectOSAuthFailureSpike deleted from the starter-alerts block; the Auth failures row deleted from the signal table above it; one paragraph added next to #86's cold-start paragraph, in that established shape
content/docs/resources/faq.mdx"auth failure rate" removed from "What metrics should I monitor?"; one clause added pointing at the Auth audit view

Re-read both neighbourhoods before editing, as instructed: #86 landed in faq.mdx (this same answer, replacing "kernel cache miss rate" with readiness) and in the observability.mdx signal table (replacing the cache row with readiness) earlier today. Both edits sit on top of that, neither reverts it. English only; locale siblings go stale and are re-derived by the translation pass.

The other half of #87 is filed, not answered

Whether the runtime should emit an auth-failure metric is an objectstack roadmap question, not an objectos docs one. Filed unassigned with the evidence as objectstack-ai/objectstack#9623. If it lands, transcribing the name back here is a one-line follow-up.

Verification

All on fe78131, the head of this branch.

CheckResult
turbo run type-check --forcecache bypass, force executing 8c984641fb51da8e · 1 successful, 0 cached
turbo run build --forcecache bypass, force executing ade11cd4db19e8fa · 1 successful, 0 cached · 550+ doc paths prerendered
turbo run test0 tasks — the docs app declares no test script (same as CI)
check-translation-ownership.mjspassed · 0 translation artifacts touched, 2 other files
check-translations.mjs✓ translations gate passed · both pages now listed stale, which is reported-not-blocking by design
check-translation-output.mjs --self-test✓ 20 case(s), every rule demonstrated able to fail
check-translation-output.mjs --files✓ gate passed · 110 pre-existing corpus findings, none in these two pages

Rendered HTML read for both pages against a real dev server, not inferred from the source:

  • /docs/operate/observability — 200. The alert block now renders ObjectOS5xxSpike alone; the signal table renders 4 rows with no auth row; auth_failures_total and ObjectOSAuthFailureSpike each occur 0 times in the served HTML.
  • /docs/resources/faq — 200. The answer renders as "5xx rate, p95 latency, readiness (/api/v1/ready), queue depth. There is no auth-failure metric to monitor — review sign-in activity in the Console's Auth audit view instead."; "auth failure rate" occurs 0 times.

Out of scope, reported not edited


Generated by Claude Code

`auth_failures_total` is emitted by nothing. On objectstack@origin/main a
repository-wide grep returns zero files (control: `SEMCONV`, 15 files), and
`packages/observability/src/semconv.ts` — the canonical registry, "so hosts
can wire alerts/dashboards against a stable namespace" — declares 13 names
covering HTTP, storage, cache, jobs and the package registry, with no auth
metric among them. A Prometheus rule on a series that is never scraped does
not error; it stays green forever, which is worse than no alert.
The card offered a third option beyond delete-or-wait: express the alert
against `http_requests_total{status=~"40[13]"}`. Measured rather than
adopted, and it does not honestly cover what the alert claimed:
- The sign-in endpoints do not feed that counter at all. `http_requests_total`
has exactly one emitter, `instrumentRouteHandler`, and it is applied by a
Proxy that `dispatcher-plugin.ts` builds over a LOCAL `server` binding and
never re-registers. plugin-auth mounts its routes on the raw Hono app
(`getRawApp()` → `rawApp.all(`${basePath}/*`)`), so a failed password on
`/api/v1/auth/sign-in/email` increments nothing.
- A rejected API key is indistinguishable from no credential. `core/src/
security/api-key.ts` validates fail-closed — "anything ambiguous (missing,
revoked, expired, malformed) resolves to no principal" — so both land on
the same anonymous 401.
- 401/403 are dominated by non-auth-failure traffic: 403 is this runtime's
PERMISSION_DENIED code (export permission, share links, tenancy override,
docs permission sets, the ADR-0069 auth gate — all users who authenticated
perfectly well), and 401 is every ordinary anonymous request.
So the alert is deleted rather than re-expressed, and the two places that
recommended the signal are corrected with it: the signal table above the
alert block, and faq.mdx's "What metrics should I monitor?". Both now point
at the `Auth` audit view, which is real — plugin-audit writes `login` /
`logout` rows and audit-logs.mdx already documents the view.
Whether the runtime SHOULD emit an auth-failure metric is an objectstack
roadmap question, not an objectos docs one; filed separately and linked from
the PR. Note the audit ledger records successful sessions only, so it is not
a substitute for that metric either.
English only; locale siblings go stale and are re-derived by the translation
pass.
@os-zhuang
os-zhuang marked this pull request as ready for review August 18, 2026 13:42
@os-zhuang
os-zhuang merged commit 6fca5d1 into mainAug 18, 2026
2 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

auth_failures_total is a metric the runtime does not emit, and observability.mdx alerts on it

2 participants

@os-zhuang@claude