rdscheck

• Copy command will:
  • Copy snapshot(s) to a different AWS region in the same account. This will copy only automated snapshots.
  • Cleanup old snapshots based on retention setup in the yaml config file
• Check command will:
  • Creates new rds instance(s) with the snapshots
  • Runs a set of queries on the database to validate the content of the backup

TODO

• Handle things gracefully when there is more than 5 snapshots to copy
• Handle different retentions between automatic and manual backups. (tag automatic snapshot with something like "CopiedBy" "rdscheck" and skip if set)

check: state machine diagram

yaml configuration file

• instances: all the rds instances that we want to copy/restore/check to an AWS region.
  • name: the name of the source rds instance
  • database: the name of the databse that we copied and restored we use this field to initiate the db connection
  • type: the rds instance type we want to use to restore the snapshot
  • password: the password that we will use to connect to the database. It doesn't need to be the original one. We will use this one to reset the original password
  • retention: how many days we want to keep the copied snapshot around. Right now it should be equal to the number of days the automatic backups are kept
  • destination: the aws region where we will copy/restore the snapshot
  • kmsid: the id (ARN) of the kms key that you want to use on the destination region. This is needed if your original snapshot is encrypted
  • queries: all the sql queries we want to run on the restored snapshot to validate it and the expected results as regex
    • query: the sql query to run
    • regex: the regex of the expected result

Example:

instances:
- name: rdscheckdatabase: rdschecktype: db.t2.micropassword: thisisatestretention: 1destination: us-east-1kmsid: "arn:aws:kms:us-east-1:1234567890:key/123456-7890-123456"queries:
- query: "SELECT tablename FROM pg_catalog.pg_tables;"regex: "^pg_statistic$"
- name: rdscheck2database: rdschecktype: db.t2.micropassword: thisisatestretention: 10destination: us-east-2queries:
- query: "SELECT tablename FROM pg_catalog.pg_tables;"regex: "^pg_statistic$"

Releases

Github Workflow is setup to create a new release when a tag is created and pushed. .github/workflows/release.yml will get triggered, will create a new release, build the commands and upload them as two seperate zip files in the release. By doing so we can then download the command zip file for a release and use it when creating a lambda function with terraform.

Terraform

module"rdscheck-copy" {
source="github.com/techdroplabs/rdscheck//terraform?ref=v0.0.9"release_version="v0.0.9"command="copy"lambda_env_vars {
variables={
S3_BUCKET ="s3-bucket-with-yaml-file"
S3_KEY ="rdscheck.yml"
AWS_REGION_SOURCE ="us-west-2"
DD_API_KEY ="lked78t4iuhweoih8oi"
DD_APP_KEY ="lknsdc8754liwhefp90"
}
}
}

module"rdscheck-check" {
source="github.com/techdroplabs/rdscheck//terraform?ref=v0.0.9"lambda_rate="rate(30 minutes)"release_version="v0.0.9"command="check"subnet_ids=["subnet-12345,subnet-6789"]
security_group_ids=["sg-1234,sg-5678"]
lambda_env_vars {
variables={
S3_BUCKET ="s3-bucket-with-yaml-file"
S3_KEY ="rdscheck.yml"
AWS_REGION_SOURCE ="us-west-2"
AWS_SG_IDS ="sg-1234,sg-5678"
AWS_SUBNETS_IDS ="subnet-qwerty1234576,subnet-azerty123456"
DD_API_KEY ="lked78t4iuhweoih8oi"
DD_APP_KEY ="lknsdc8754liwhefp90"
}
}
}