Skip to content

Monitoring, alerting, and on-call runbook #133

Description

@truthixify

Tier: L (5-7 days) | Type: docs

Context. guides/ops/self-hosted-deployment.mdx (Wave 7) walks getting contracts up. Nothing today tells the operator what to watch, when to page, or how to respond. Teams running Wraith in production need a full runbook: healthy metric baselines, ready-to-import dashboards, alert rules, incident playbooks, and a tabletop exercise. This is more than a page; it is a bundle of ops artifacts, hence the larger tier.

Scope.

  • New guides/ops/monitoring-and-on-call.mdx.
  • Key metrics: RPC latency, announcement-lag, indexer backlog, watcher event-drop rate, contract error rate.
  • Grafana dashboard JSON in guides/ops/dashboards/ (one for RPC, one for indexer, one for contracts).
  • Prometheus alerting rules file with rationale per rule.
  • Six named incident playbooks: RPC outage, Horizon backpressure, indexer stall, contract mispublish, watcher drop-spike, key-rotation incident.
  • Sev matrix mapping incident types to response SLA, cross-linked with the auditor guide's severity ladder.
  • Tabletop exercise script for a quarterly drill.

Acceptance.

  • Reviewer imports each dashboard into a Grafana instance without error
  • Alert rules validated with promtool check rules
  • Reviewer walks one playbook end-to-end on a staging deployment
  • Renders in mint dev
  • Passes Compile docs snippets CI
  • Cross-linked with self-hosted-deployment.mdx and reference/auditor-guide.mdx
  • Nav entry under Operations

Files. guides/ops/monitoring-and-on-call.mdx (new), guides/ops/dashboards/ (new), guides/ops/alerts/ (new), docs.json.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programdocsDocumentationdripsFunded via Drips Networkhelp wantedExtra attention is neededopsstellar-waveAuto-created for Wave 8wave-8Auto-created for Wave 8writing

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions