feat(spec): IHttpServer gains an afterResponse response-observing hook — HTTP metrics become transport-agnostic

.changeset/http-server-response-observation-seam.md

@@ -0,0 +1,45 @@
+---
+"@objectstack/spec": minor
+"@objectstack/core": patch
+"@objectstack/observability": patch
+"@objectstack/plugin-hono-server": patch
+"@objectstack/runtime": patch
+"@objectstack/http-conformance": patch
+---
+
+feat(spec): `IHttpServer` gains an optional `afterResponse` response-observing
+hook so HTTP metrics are transport-agnostic instead of Hono-only (#9835)
+
+The contract addition (additive — a new optional member plus the
+`HttpResponseObservation` / `HttpResponseObserver` types and the reserved
+`UNMATCHED_ROUTE_PATTERN` label): a transport invokes each registered observer
+exactly once per answered request with `{ method, routePattern, status,
+elapsedMs }`, after the response exists — the observation point the `use()`
+middleware contract cannot express (it runs before dispatch and never sees a
+status). `routePattern` is REQUIRED to be the registered route pattern
+(`/api/v1/data/:id`), never the concrete path, so no adapter re-decides metric
+cardinality. Optionality is feature-detected runtime-real
+(`typeof server.afterResponse === 'function'`); a transport that does not
+implement the seam reports **no** HTTP metrics — zero there means "not
+instrumented", never "no traffic".
+
+Implementations and consumers in the same change:
+
+- `@objectstack/plugin-hono-server`: `HonoHttpServer` implements the seam (the
+ ruled #9650 raw-app middleware becomes its delivery path — same reach,
+ including `getRawApp()` mounts and middleware-refused 429s); unrouted
+ requests are now labelled with the reserved `unmatched` pattern (previously
+ they could surface as `/*`).
+- `@objectstack/observability`: new `armHttpRequestCounter(server, metrics)`
+ arms the `http_requests_total` counter through the seam at most once per
+ server (first caller wins), which is what makes "exactly one counter per
+ server" structural.
+- `@objectstack/runtime`: the dispatcher offers its `observability.metrics`
+ registry to the seam (a host that wires only the dispatcher now counts every
+ inbound surface) and suppresses its own per-route copy of
+ `http_requests_total` when the transport implements the seam — retiring the
+ #9833 double count. Request-id echo, the duration histogram, the error
+ counter and the error reporter are unchanged.
+- `@objectstack/http-conformance`: `NodeHttpServer` implements the seam, and a
+ new cross-adapter conformance suite locks the semantics for both adapters.
+- `@objectstack/core`: re-exports the new contract types/constant.

content/docs/deployment/production-readiness.mdx

@@ -55,11 +55,24 @@ That's it — every route the dispatcher mounts now:
 
 1. Sends conservative security headers.
 2. Echoes `X-Request-Id` (honored from caller, or freshly minted).
-3. Emits `http_requests_total{method,route,status}` and
- `http_request_duration_ms` metrics.
+3. Emits the `http_request_duration_ms` histogram.
 4. Reports 5xx exceptions to Sentry (or your reporter) with the
  request id attached.
 
+The `http_requests_total{method,route,status}` counter is emitted at the
+**transport** (the `IHttpServer.afterResponse` observation seam), so it covers
+*every* inbound request on the server — auth, the REST data API and other
+raw-app mounts included, not only the dispatcher's own routes. The dispatcher
+offers its `metrics` registry to that seam automatically, and exactly one
+counter is armed per server, so the wiring above is complete on its own and
+wiring the transport plugin too never double-counts. The `route` label is the
+registered pattern (`/api/v1/data/:id`), never the concrete path.
+
+**A transport that does not implement the seam reports no HTTP metrics**: zero
+on `http_requests_total` there means "not instrumented", never "no traffic".
+The shipped Hono adapter implements it; verify a custom adapter with
+`typeof server.afterResponse === 'function'` before trusting its numbers.
+
 Rate limiting is the one piece you wire at the adapter layer (Fastify
 preHandler, Hono middleware, etc.) because that's where you have
 reliable access to the caller's IP and authenticated identity. See

docs/OBSERVABILITY.md

@@ -9,11 +9,28 @@
 `createDispatcherPlugin` automatically instruments every route it mounts with:
 
 - **Request id** propagation: honors incoming `X-Request-Id` (or mints `req_<uuid>`); echoes on the response.
-- **`http_requests_total{method,route,status}`** counter (1 per request).
 - **`http_request_duration_ms{method,route}`** histogram (handler latency).
 - **`http_request_errors_total{method,route}`** counter (incremented on thrown errors).
 - **Error reporting** for 5xx (handler-thrown or via `errorResponseBase` side channel).
 
+**`http_requests_total{method,route,status}`** (1 per request) is emitted at the
+**transport**, through the `IHttpServer.afterResponse` observation seam — so it
+counts *every* inbound request on the server (auth, REST data API, raw-app
+mounts, requests a middleware refused with 429), not only the routes the
+dispatcher registers. The `route` label is always the registered **pattern**
+(`/api/v1/data/:id`), never the concrete path; requests no route matched are
+labelled `unmatched`. Exactly one counter is armed per server (first wiring
+wins), so handing one registry to both the transport plugin and the dispatcher
+never double-counts.
+
+> **A transport that does not implement the `afterResponse` seam reports no
+> HTTP metrics.** On such a transport `http_requests_total` stays flat while
+> traffic flows: zero there means "not instrumented", never "no traffic". Ask
+> with `typeof server.afterResponse === 'function'` before reading absence as
+> coverage. (The dispatcher degrades on those transports by counting its own
+> routes, which is all it can see.) The shipped Hono adapter and the
+> `@objectstack/http-conformance` node adapter both implement the seam.
+
 Defaults are no-op (zero overhead). Inject real adapters via the `observability` config:
 
 ```ts

packages/core/src/index.ts

@@ -88,10 +88,15 @@ export type {
  IHttpResponse,
  RouteHandler,
  Middleware,
+ HttpResponseObservation,
+ HttpResponseObserver,
  IDataEngine,
  IObjectQLEngine,
  EngineSchemaRegistryView,
  EngineTransactionOptions,
  EngineTransactionInfo,
  IDataDriver,
 } from '@objectstack/spec/contracts';
+// The reserved route label for unrouted requests on the `afterResponse`
+// observation seam (#9835) — a VALUE, so it rides beside the type block above.
+export { UNMATCHED_ROUTE_PATTERN } from '@objectstack/spec/contracts';

packages/observability/src/__tests__/http-transport-metrics.test.ts

@@ -0,0 +1,106 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import { describe, it, expect } from 'vitest';
+import type { IHttpServer, HttpResponseObserver } from '@objectstack/spec/contracts';
+import { armHttpRequestCounter } from '../http-transport-metrics.js';
+import { InMemoryMetricsRegistry } from '../metrics-exporters.js';
+
+/** A minimal `IHttpServer` whose `afterResponse` we can drive by hand. */
+function observableServer() {
+ const observers: HttpResponseObserver[] = [];
+ const server: IHttpServer = {
+ get: () => {},
+ post: () => {},
+ put: () => {},
+ delete: () => {},
+ patch: () => {},
+ use: () => {},
+ listen: async () => {},
+ afterResponse: (observer) => {
+ observers.push(observer);
+ },
+ };
+ const deliver = (routePattern: string, status: number) => {
+ for (const observer of observers) {
+ observer({ method: 'GET', routePattern, status, elapsedMs: 1 });
+ }
+ };
+ return { server, observers, deliver };
+}
+
+describe('armHttpRequestCounter (#9835)', () => {
+ it('arms the counter through the afterResponse seam, labelled by PATTERN with stringified status', () => {
+ const { server, deliver } = observableServer();
+ const metrics = new InMemoryMetricsRegistry();
+
+ expect(armHttpRequestCounter(server, metrics)).toBe('armed');
+ deliver('/api/v1/data/:id', 200);
+ deliver('/api/v1/data/:id', 503);
+
+ expect(
+ metrics.totalCounter('http_requests_total', {
+ method: 'GET',
+ route: '/api/v1/data/:id',
+ status: '200',
+ }),
+ ).toBe(1);
+ expect(
+ metrics.totalCounter('http_requests_total', {
+ route: '/api/v1/data/:id',
+ status: '503',
+ }),
+ ).toBe(1);
+ });
+
+ it('latches per server — a second arming call registers NOTHING, whoever brings the registry', () => {
+ // The contract's one-owner rule made structural: the transport plugin
+ // (Phase 1) and the dispatcher's registry offer (Phase 2) both route
+ // through here; first caller wins and one registry handed to both
+ // layers can never double-count (#9833's shape, closed).
+ const { server, observers, deliver } = observableServer();
+ const first = new InMemoryMetricsRegistry();
+ const second = new InMemoryMetricsRegistry();
+
+ expect(armHttpRequestCounter(server, first)).toBe('armed');
+ expect(armHttpRequestCounter(server, first)).toBe('already-armed');
+ expect(armHttpRequestCounter(server, second)).toBe('already-armed');
+ expect(observers).toHaveLength(1);
+
+ deliver('/counted', 200);
+ expect(first.totalCounter('http_requests_total', { route: '/counted' })).toBe(1);
+ // The later registry was refused, not silently added.
+ expect(second.totalCounter('http_requests_total', { route: '/counted' })).toBe(0);
+ });
+
+ it('reports "unsupported" for a transport without the seam and registers nothing', () => {
+ const server: IHttpServer = {
+ get: () => {},
+ post: () => {},
+ put: () => {},
+ delete: () => {},
+ patch: () => {},
+ use: () => {},
+ listen: async () => {},
+ };
+ const metrics = new InMemoryMetricsRegistry();
+
+ // The documented expectation, surfaced instead of read as coverage:
+ // this transport reports NO HTTP metrics; the caller decides how to
+ // degrade (the dispatcher falls back to counting its own routes).
+ expect(armHttpRequestCounter(server, metrics)).toBe('unsupported');
+ expect(metrics.samples).toHaveLength(0);
+ });
+
+ it('latches per SERVER, not per process — a second server arms independently', () => {
+ const a = observableServer();
+ const b = observableServer();
+ const metrics = new InMemoryMetricsRegistry();
+
+ expect(armHttpRequestCounter(a.server, metrics)).toBe('armed');
+ expect(armHttpRequestCounter(b.server, metrics)).toBe('armed');
+ a.deliver('/a', 200);
+ b.deliver('/b', 200);
+ expect(metrics.totalCounter('http_requests_total', { route: '/a' })).toBe(1);
+ expect(metrics.totalCounter('http_requests_total', { route: '/b' })).toBe(1);
+ });
+});

packages/observability/src/http-transport-metrics.ts

@@ -0,0 +1,82 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import type { IHttpServer } from '@objectstack/spec/contracts';
+import type { MetricsRegistry } from './contracts.js';
+import { RUNTIME_METRICS } from './semconv.js';
+
+/**
+ * What {@link armHttpRequestCounter} did:
+ *
+ * - `'armed'` — the counter-emitting observer was registered on this call.
+ * - `'already-armed'` — some earlier caller already armed this server; the
+ * seam counts, and this call registered nothing (first-wins).
+ * - `'unsupported'` — the transport does not implement the
+ * `IHttpServer.afterResponse` seam; it reports NO HTTP metrics (#9835:
+ * zero there means "not instrumented", never "no traffic"), and the
+ * caller must decide how to degrade — the runtime dispatcher falls back
+ * to counting its own routes.
+ */
+export type ArmHttpRequestCounterResult = 'armed' | 'already-armed' | 'unsupported';
+
+/**
+ * The per-server latch behind the contract's ownership rule. A registered
+ * global-registry symbol (`Symbol.for`), not a module-level WeakSet, so the
+ * latch cannot fork if two copies of this module ever coexist in one
+ * process — the whole point is that there is exactly ONE latch per server
+ * object, whoever asks.
+ */
+const HTTP_REQUEST_COUNTER_ARMED = Symbol.for(
+ 'objectstack.observability.httpRequestCounterArmed',
+);
+
+/**
+ * Arm `http_requests_total{method,route,status}` on a transport through the
+ * `IHttpServer.afterResponse` observation seam (#9835) — AT MOST ONCE per
+ * server, whoever calls first.
+ *
+ * ## Why arming is centralized here
+ *
+ * The contract's ownership rule says a request must never be double-counted,
+ * and two composition layers legitimately hold both a server and a metrics
+ * registry: the transport's own hosting plugin (`HonoServerPlugin`, which
+ * the 2026-08-18 ruling on #9650 made the counter's home) and the runtime
+ * dispatcher (whose `observability.metrics` config is the wiring the docs
+ * demonstrate). When a host hands ONE registry to both — the ordinary case —
+ * two independently-registered observers would land every request on the
+ * same series twice: exactly the #9833 distortion, rebuilt one seam over.
+ * Routing every arming through this function makes "exactly one
+ * counter-emitting observer per server" structural: the first caller arms
+ * (in the shipped composition that is the transport plugin, in Phase 1),
+ * every later caller is told the seam already counts.
+ *
+ * The label shape is pinned by the contract: `route` is the transport's
+ * `routePattern` — the registered PATTERN, never the concrete path — and
+ * `status` is stringified for the label set. Emission goes through the
+ * transport's observer-isolation guarantee, so a throwing registry cannot
+ * break a response.
+ *
+ * @param server - The transport. Pass the RAW registered `http.server`
+ * instance, not a wrapper: the latch is per object identity, and a wrapper
+ * would both fork the latch and (per #5122) risk erasing the optional
+ * member this function feature-detects.
+ * @param metrics - The registry the counter lands in. First caller wins; a
+ * second registry offered later is NOT added (the contract's one-owner
+ * rule), and the result says so.
+ */
+export function armHttpRequestCounter(
+ server: IHttpServer,
+ metrics: MetricsRegistry,
+): ArmHttpRequestCounterResult {
+ if (typeof server.afterResponse !== 'function') return 'unsupported';
+ const latched = server as IHttpServer & { [HTTP_REQUEST_COUNTER_ARMED]?: boolean };
+ if (latched[HTTP_REQUEST_COUNTER_ARMED]) return 'already-armed';
+ latched[HTTP_REQUEST_COUNTER_ARMED] = true;
+ server.afterResponse((observation) => {
+ metrics.counter(RUNTIME_METRICS.httpRequestsTotal, {
+ method: observation.method,
+ route: observation.routePattern,
+ status: String(observation.status),
+ });
+ });
+ return 'armed';
+}

packages/observability/src/index.ts

@@ -16,6 +16,13 @@ export { OBSERVABILITY_METRICS_SERVICE, OBSERVABILITY_ERRORS_SERVICE } from './s
 // Semantic conventions
 export { SEMCONV, RUNTIME_METRICS } from './semconv.js';
 
+// Transport-agnostic HTTP request counter, armed at most once per server via
+// the `IHttpServer.afterResponse` observation seam (#9835)
+export {
+ armHttpRequestCounter,
+ type ArmHttpRequestCounterResult,
+} from './http-transport-metrics.js';
+
 // Metric exporters
 export {
  NoopMetricsRegistry,