Uh oh!
There was an error while loading. Please reload this page.
Surface a licensed max_nodes oversell to operators as telemetry (visibility only) - #12720
Conversation
…perator telemetry Extends the reach of the boot-time cap advisory: the same advisory verdict is now also emitted through the deployment's configured metrics backend, so an operator can ask the question and alert on it long after the boot log scrolled away. Visibility only. The gate stays advisory, nothing is refused, and the surface is named around what the operator DECLARED and what the licence ADMITS — this process has no cluster membership view at all.
📓 Docs Drift CheckThis PR changes 3 package(s): 17 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 26 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin a04656bf99a737bd880d73a7971de5b90583832d && git checkout a04656bf99a737bd880d73a7971de5b90583832d
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 4af6c4419d7669a56940ec8e7adbb133829fcbde b7d45784cebf6fb25d7ff66c4921b999555e9474 && git checkout -B drift-repro 4af6c4419d7669a56940ec8e7adbb133829fcbde && git merge --no-ff b7d45784cebf6fb25d7ff66c4921b999555e9474
node scripts/docs-audit/affected-docs.mjs --json 4af6c4419d7669a56940ec8e7adbb133829fcbde
|
Uh oh!
There was an error while loading. Please reload this page.
Fixes#12667
Maintainer ruling 2026-08-27, quoted unchanged: 「其他接受」 — adopting option C on #8501. A licensed
max_nodesoversell becomes visible to operators; the atomic slot-claim enforcement mechanism is deliberately not built here.Where the reading can land — measured first, before anything was written
The card asked this to be settled before designing, and it is the decision the diff follows from.
packages/consolein this repo is a prebuilt SPA and nothing else: itsexportsmap contains only./package.json, itsfileslist isdist, and its README states the source of truth is@object-ui/consoleinobjectstack-ai/objectui, frozen at the SHA in.objectui-sha. There is no console source here to add a panel to. Per the card's own instruction that half is reported rather than absorbed.@objectstack/observabilityowns the metric-name conventions (SEMCONV) and theMetricsRegistrycontract;os servealready builds and registers a backend fromOS_OBS_*. The reading needed no new plumbing, and no new public route or client method — clause ② staysno.Premises, re-measured on today's tree
All three still hold after the seam moved at #8503, and together they are why this is visibility rather than enforcement:
checkMultiNodeAllowed(...)call inpackages/cli/src/commands/serve.ts, reached only whenOS_CLUSTER_DRIVERnames a remote driver;generateNodeId()inpackages/services/service-cluster/src/cluster.tsisMath.random-seeded per process, and no join/leave registry exists;OS_CLUSTER_REPLICASis an operator-declared count, identical in every replica —declaresMultiNode()insplit-brain-guard.tsreads it as a declaration, never as an observation.So nothing in the process knows how many peers exist, and the surface must not read as though it does.
What this adds
Three names join
SEMCONVin@objectstack/observability, emitted once per boot byos serve, each labelled with the verdict vocabulary #8367 / PR #8503 landed (admitted/capped/refused) rather than a second vocabulary:cluster_declared_nodescluster_admitted_nodescluster_node_cap_verdicts_totaldescribeMultiNodeCapTelemetry()sits beside the existingformatMultiNodeCapAdvisory()inserve.ts— the same consumer, extending the same advisory's reach rather than duplicating the #8504 boot warning, which is untouched and still fires. Deliberately kept a pure function so the tests can assert what an operator receives.Three omissions are load-bearing, because each alternative would be a false number rather than a missing one: no declared gauge when nothing was declared (
0would read as a declaration of zero replicas); no admitted gauge when the gate expressed no cap (any number invents a limit nobody stated); nothing at all when the gate was never consulted.The counter exists because the exporter is push-based with no gauge re-observation, so a one-shot gauge ages out of the backend —
increase(cluster_node_cap_verdicts_total{verdict="capped"}[1h]) > 0stays alertable after it has.What this explicitly does NOT add
No enforcement, no membership view, no slot claim, no refusal behaviour. The cap stays advisory: every declared replica still joins, exactly as before. Nothing published gains a new contract, and no accept/reject behaviour changed anywhere.
The honest-naming assertion the card asked for is pinned in
serve-multi-node-cap-telemetry.test.ts: every emitted metric name, label name and label value is swept against a membership-claim vocabulary (running,active,live,members,peers,observed, …), with a vacuity proof that the sweep rejectscluster_active_nodesand acceptscluster_declared_nodes. That vacuity proof earned its place immediately — the sweep was first written with\banchors, which do not fire insidecluster_active_nodesbecause_is a word character, so the rename this guard exists to reject would have sailed through green. The anchors are now explicit and the measurement is recorded in the file.Verification
Gate union derived from the actual changed set with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, re-run on the final commitb7d45784c: 24 green. One was not measured —scripts/pm/check-half-states.mjsexits 3 withPREREQUISITE NOT METbecause this container holds no valid GitHub credential; that is no reading either way, not a failure.pnpm check:type-check-debt --re-measurere-measured 31 ledger entries against a fully built workspace: none above its recorded number.check:slot-lookupcaught a real defect in the first draft of this change — the metrics lookup was assigned to anany-declared variable, growingserve.ts's erasure count 9 to 10. Typing itMetricsRegistryfixed the ratchet and also made the emission calls type-checked, which matters here:counter(name, labels, value)andgauge(name, value, labels)take their arguments in different orders, and underanya swap would have compiled and emitted garbage.Suites:
@objectstack/cli(the four multi-node / observability files) 41 passed;@objectstack/observability85 passed;@objectstack/service-cluster66 passed;pnpm --filter @objectstack/cli typecheckclean — and confirmed withtsc --noEmit --listFilesthat both new test files are genuinely inside that program rather than excluded from it.Ablation, two legs, each proved on disk before its colour was read:
'admitted'. Removed-text count 0, injected-marker count 1,git diffnon-empty. Result: 3 of 17 red, on exactly the verdict rows. This leg involves nodist: the suite imports./serve.jsby relative path inside the same package, so no build state can hide it.cluster_declared_nodesrenamed tocluster_active_nodesinSEMCONV. Becauseserve.tsconsumes that through@objectstack/observability'sexports(dist, not src), the package was rebuilt andscripts/ablation-dist-preflight.mjsconfirmed the marker present in 4 built files before the run. Result: 8 of 17 red, including the membership sweep itself.Both restore legs used
git checkout HEAD -- absolute-pathunder anEXIT/INT/TERMtrap, and are proved by state rather than exit code: emptygit diff HEAD, emptygit status --porcelain, and blob hashes identical to theirHEADblobs. The dist leg was rebuilt on restore too, and its--absentpre-flight passes for a marker unique to the mutation. Worth recording: the first--absentcheck reported a hit and it was not a failed restore —cluster_active_nodeslegitimately appears in the newSEMCONVdoc comment naming it as a forbidden spelling, and tsup preserves comments intodist. The mutated value was absent from source and dist alike. A marker that also occurs in the honest source cannot answer this question; that is a lesson about marker choice, not about the restore.Changeset:
.changeset/lucky-donkeys-vanish.md, minor on@objectstack/observabilityand@objectstack/cli— new metric names in a publishedSEMCONVand a new emission from a shipped command are both user-visible.Refs: #8501 (the ruling and its four-facet record), #8367 and PR #8503 (the verdict seam), #8504 (the boot warning this extends, untouched), objectstack-ai/cloud#1275 (the 2026-08-13 overflow-semantics ruling, unchanged). None of those is addressed here.
Generated by Claude Code