Decide whether SEMCONV should declare an auth-failure metric — nothing instruments authentication today, and 401/403 out of `http_requests_total` cannot stand in for it

[os-zhuang]

Split out of objectstack-ai/objectos#87, which found that content/docs/operate/observability.mdx shipped a Prometheus alert on auth_failures_total — a metric nothing emits. That card asked two questions and only one belonged to the docs repo: what should the page say today (answered there — the alert is deleted) and should the runtime emit an auth-failure metric at all, which is this one. Filed rather than answered, unassigned.

The measurement

On origin/main at b057e53f4:

git grep auth_failures_total origin/main → 0 files
git grep -l SEMCONV origin/main → 15 files (control: the grep and the tree are fine)

packages/observability/src/semconv.ts is the canonical registry — in its own words, it exists "so hosts can wire alerts/dashboards against a stable namespace and so call sites don't sprinkle string literals through the code base". It declares 13 names, grouped by the subsystem that emits them:

Group	Names
HTTP	http_requests_total, http_request_duration_ms, http_request_errors_total
Storage	storage_operations_total, storage_operation_duration_ms, storage_errors_total
Cache	cache_lookups_total, cache_writes_total, cache_errors_total
Jobs	job_schedule_failures_total
Registry	registry_lookups_total, registry_lookup_duration_ms, registry_source_fetches_total

No auth metric among them. Authentication is the one inbound subsystem in that list's shape that instruments nothing.

Why the obvious substitute does not work

The natural reply is "count 401/403 out of http_requests_total". I measured it while fixing the docs, and it does not cover what an auth-failure alert claims — three independent reasons, any one of which is disqualifying:

1. The sign-in endpoints do not feed that counter at all. http_requests_total has exactly one emitter, instrumentRouteHandler (packages/runtime/src/observability/instrument.ts:106). It is applied by a Proxy that packages/runtime/src/dispatcher-plugin.ts:700-721 builds over a local server binding obtained from ctx.getService('http.server'); the proxy is never registered back as the service. plugin-auth mounts its routes on the raw Hono app instead — registerAuthRoutes calls getRawApp() and then rawApp.all(basePath + '/*') (packages/plugins/plugin-auth/src/auth-plugin.ts:1622,2240). So a failed password on /api/v1/auth/sign-in/email increments nothing. (packages/rest also resolves http.server itself, at rest-api-plugin.ts:129, so the REST data API is outside the counter too — worth its own look, but it is not what this card is about.)

2. A rejected API key is indistinguishable from no credential. packages/core/src/security/api-key.ts validates fail-closed by design: "anything ambiguous (missing, revoked, expired, malformed) resolves to no principal, never to an elevated one". A presented-and-rejected key and a request that carried nothing both arrive at the route as anonymous, and both answer 401.

3. 401/403 are dominated by traffic that is not an auth failure. 403 is this runtime's authorization code — PERMISSION_DENIED, export-permission denials, share-link denials, NOT_OVERRIDABLE tenancy overrides, docs permission sets, and the ADR-0069 auth gate (password expiry / enforced MFA), all of which are raised against users who authenticated perfectly well. 401 is every ordinary anonymous request: a signed-out tab polling an API route, an expired session.

The audit ledger is not a substitute either: plugin-audit's AuthSessionAuditEvent is a closed union of login | logout with a required userId ("a real sys_user id"), so it records successful sessions only. There is no failed-sign-in row anywhere.

Why it would belong in SEMCONV

Whether to build it is the decision here, not mine. The case for, briefly: sustained auth-failure rate is the standard first-line signal for credential stuffing and for a misconfigured IdP, it is the one thing an operator asks for that this registry cannot answer, and every other inbound subsystem in the registry is already instrumented at its own seam rather than inferred from HTTP status. If it lands, the natural shape mirrors the existing entries — a counter with a label distinguishing the mechanism that rejected (password / api-key / oidc / mfa) — and semconv.ts's own doc comment is where the name would be declared.

If the answer is "no", that is equally actionable: the docs now say plainly that no such metric exists, and this card can close as not-planned so the next reader of observability.mdx does not re-open the question.

Note on filing

search_issues returns 0 for every query against this repository right now, including a control that must match (open #9621, "sanitizer" in its title). I checked for duplicates by listing all 229 open issues instead; none covers this.

Related: objectstack-ai/objectos#87, objectstack-ai/objectos#70.

---

Generated by Claude Code

[os-support-ai]

Triage: decision inbox (needs-user-decision), type Feature, routed domain:cli for eventual landing (packages/observability SEMCONV + the emitting seam). Declaring a new metric name expands the declared public surface — manual floor, not adjudicable.

Four-prism block:

1. Platform long-term coherence — the registry's own shape argument is real: every other inbound subsystem instruments at its own seam; auth is the one that infers from HTTP status, and the card proves the inference is triply broken.
2. Measured business pull — zero today: the only "demand" was a docs alert someone wrote aspirationally (now deleted). No deployment has asked for credential-stuffing detection yet.
3. AI-agent error-resistance — neutral-to-positive: a declared counter with a closed mechanism label (password | api-key | oidc | mfa) is a closed enumeration; but declaring it obligates wiring every rejection path, or ADR-0049 makes it a declared-unenforced violation on day one.
4. Startup scope discipline — this is capability expansion with no pull; declare-and-maintain is a permanent obligation across every auth path present and future.

Recommendation: not now. Close not planned (docs are already honest that no such metric exists; this card then stands as the recorded answer for the next reader of observability.mdx), revisit on first real operator pull. If the maintainer wants it anyway, contract-first: SEMCONV name + mechanism label enum first, wiring in plugin-auth's rejection paths second, and the http_requests_total blind spot stays a separate card either way.

Side-product: the card's §1 measurement (auth routes and the REST data API mount outside the http_requests_total proxy) is a distinct, larger observability defect — filed separately as a finding by this seat (see cross-reference on that card).

---

Generated by Claude Code

[os-support-ai]

Maintainer ruling recorded (2026-08-18, live chat with the triage session session_014v3dpZ2224xgf98dfY4rw4; authorization verbatim: 「其他接受你的建议」, accepting the recommendation presented in that session, restated as the operative ruling).

Ruling: not planned now — closing. No auth_failures_total (or auth-metric family) is added to SEMCONV today: it is a capability expansion with zero measured deployment pull, and the docs half already tells operators plainly that no such metric exists. Per this card's own framing, "no" is equally actionable.

Recorded restart conditions (either reopens this card — it stays the anchor):

1. After http_requests_total never sees auth routes or the REST data API — the two highest-traffic inbound surfaces are outside the only HTTP counter the docs tell operators to monitor #9650's ruled seam lands (metric emission moves into the Hono adapter, pattern-labeled with status): re-evaluate whether 401-rate scoped to the auth mount patterns covers the operator need — the strongest objection here (sign-in endpoints feed no counter at all) disappears at that point, and objections 2–3 (rejected-key vs no-credential indistinguishability; 401/403 dominated by non-failure traffic) shrink to the auth-mount slice.
2. Any real deployment or security-review demand for a dedicated auth-failure signal (credential stuffing / IdP misconfiguration monitoring) — that is the measured pull this ruling waits for.

Closed as not planned; reopening is free and carries this analysis with it.

---

Generated by Claude Code