Skip to content

Instrument database pool roles - #7356

Merged
TheSentinel454 merged 5 commits into
mainfrom
tornquist/database-pool-roles
Sep 16, 2026
Merged

TheSentinel454 merged 5 commits into
mainfrom
tornquist/database-pool-roles

Conversation

@TheSentinel454

Copy link
Copy Markdown
Contributor

What

Name the relay's four Postgres pool roles—writer, reader, audit, and search—and export one fixed-cardinality utilization metric contract for all of them. Existing writer and reader gauges remain unchanged.

Why

Audit and search already use separate pools, but they are missing from pool pressure telemetry. Operators cannot see all process-level connection demand before setting a deployment-wide connection budget.

How

A closed DbPoolRole vocabulary now labels buzz_db_pool_connections{pool_role,state} and buzz_db_pool_configured{pool_role}. The relay retains cheap pool clones for statistics only; ownership, capacities, timeouts, query routing, and failure behavior do not change. The docs also replace the stale async search-index description with the generated-column behavior used today.

This simplifies four ad hoc observability states into one bounded model without adding a pool manager or changing service boundaries.

Risk

Low to moderate. This changes relay metrics and pool construction plumbing, but not SQL execution or routing. Legacy metric names remain available for current dashboards.

Testing

  • Push gate at 3f5468a0463bdfc5879bcdd3878fa06b42ca84c2: all 14 Rust test lanes passed; desktop Tauri checks passed.
  • Blox: cargo test --workspace --all-targets with repository-declared Postgres services.
  • Blox: cargo clippy --workspace --all-targets -- -D warnings.
  • Blox: cargo fmt --all -- --check.

Bigger picture

Aggregate deployment budget enforcement remains deferred. This PR exposes the per-role facts that a later deployment-policy change can consume.

Generated with Claude Code

Expose one closed physical Postgres pool-role vocabulary — writer,
reader, audit, and search — used only at pool construction and the
utilization-metrics boundary. Emit a fixed-cardinality role-labelled
contract (buzz_db_pool_connections{pool_role,state} plus
buzz_db_pool_configured{pool_role}) so all four roles are always present
and unconfigured optional pools report zero rather than disappearing.

The existing writer buzz_db_pool_* and reader buzz_db_read_pool_* gauges
are preserved unchanged for dashboard compatibility. Audit and search
keep cheap PgPool clones purely as statistics handles; service
ownership, pool capacities, timeouts, and query routing are unchanged,
and no aggregate deployment connection budget is enforced.

Document the new families in the Helm chart operator README and correct
the stale ARCHITECTURE.md claim that search indexing is an asynchronous
pipeline step — search_tsv is a generated column maintained on insert.

Signed-off-by: tornquist <tornquist@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is 6d1f488d2273dd314184fc2ec805f8982327dc51...50693d83bd8e330480252659ae0b93ee88a132cf.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review 50693d83bd8e330480252659ae0b93ee88a132cf to authorize a new review.
Any previous review applies only to its recorded range.

TheSentinel454 and others added 2 commits September 4, 2026 19:09
Signed-off-by: tornquist <tornquist@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: tornquist <tornquist@squareup.com>
@TheSentinel454
TheSentinel454 marked this pull request as ready for review September 16, 2026 06:14
@TheSentinel454
TheSentinel454 requested a review from a team as a code owner September 16, 2026 06:14
Signed-off-by: tornquist <tornquist@squareup.com>
Co-authored-by: Codex <noreply@openai.com>

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approved. I independently verified the fixed-cardinality role contract and legacy compatibility at exact head e1019fd10371e94dba91c6c744a24a6a926af220, including live relay scrapes with audit disabled and enabled.

Nit, nonblocking: consider separating size and max from the state dimension. idle and active are connection states, but size contains both and max is capacity. Consequently, an unqualified sum over buzz_db_pool_connections is not meaningful (size + idle + active + max double-counts current connections and adds capacity).

Prometheus's metric naming guidance says:

As a rule of thumb, either the sum() or the avg() over all dimensions of a given metric should be meaningful (though not necessarily useful). If it is not meaningful, split the data up into multiple metrics. For example, having the capacity of various queues in one metric is good, while mixing the capacity of a queue with the current number of elements in the queue is not.

A follow-up could retain idle|active as states and expose role-labelled current size and capacity separately. This does not block the present change: values are accurate, labels are bounded, and there are no existing consumers of the new family in the reviewed repository paths.

Signed-off-by: tornquist <tornquist@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
@TheSentinel454

Copy link
Copy Markdown
Contributor Author

Addressed the metric-semantics suggestion at 50693d83bd8e330480252659ae0b93ee88a132cf.

  • buzz_db_pool_connections{pool_role,state} now contains only the disjoint idle and active states, so an unqualified sum counts each current connection once.
  • buzz_db_pool_max_connections{pool_role} now reports capacity separately.
  • I omitted a role-labelled size metric because it is exactly idle + active.
  • buzz_db_pool_configured{pool_role} and all legacy writer/reader gauges remain unchanged.
  • The new utilization/configuration contract drops from 20 to 16 bounded series per pod.

The feedback was submitted in an approval review body, not an inline review thread, so GitHub exposes no thread to resolve. This comment records the addressed SHA.

Generated with Codex

@TheSentinel454
TheSentinel454 enabled auto-merge (squash) September 16, 2026 15:20
@TheSentinel454
TheSentinel454 merged commit 2130920 into main Sep 16, 2026
89 checks passed
@TheSentinel454
TheSentinel454 deleted the tornquist/database-pool-roles branch September 16, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants