Skip to content

Automated Snapshot Creation & README Update #56

Description

@curryxbo

Summary

Add a server-side automation system for creating chain data snapshots, uploading to S3, and updating the README snapshot table via GitHub PR.

Components

  • snapshot_make.py — main entry point, runs via PM2 cron on the 1st and 15th of each month
  • update_metadata.py — fetches indexer metadata and creates a GitHub PR to update the README table via GitHub REST API (no git/gh CLI required)
  • update_readme.py — in-memory README table insertion logic
  • metrics_server.py — Prometheus metrics HTTP server for alerting integration
  • snapshot.env.example — per-environment configuration template
  • ecosystem.config.js.example — PM2 process config template

Workflow

  1. Resolve snapshot name (check GitHub for branch conflicts)
  2. Stop morph-geth and morph-node
  3. Copy chain data → compress → upload to S3
  4. Restart services, collect base_height via geth RPC
  5. Query explorer-indexer API for l1_msg_start_height and derivation_start_height
  6. Push README update to a new branch and open a PR

Key Design Decisions

  • All GitHub operations use REST API directly — no git or gh CLI dependency
  • Snapshot name, S3 key, and branch name are resolved together before any destructive operation to prevent conflicts
  • All temp files are scoped under SNAPSHOT_WORK_DIR (cleared after upload)
  • Python 3.7 compatible
  • Supports multiple environments (mainnet / hoodi / holesky) and snapshot types (snapshot / mpt-snapshot) via env config

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions