Module 14 - Automation with Python

This repository contains a demo project created as part of my DevOps studies in the TechWorld with Nana – DevOps Bootcamp.

Demo Project: Data Backup & Restore

Technologies used: Python, Boto3, AWS

Project Description:

• Write a Python script that automates creating backups for EC2 Volumes
• Write a Python script that cleans up old EC2 Volume snapshots
• Write a Python script that restores EC2 Volumes

---

Prerequisites

• Python 3.12+ and uv for dependency management
• An AWS account with permissions to manage EC2 instances, EBS volumes and snapshots
• Boto3 — the AWS SDK for Python (EC2 client reference)
• schedule — for running the backups on a recurring basis

Install Python dependencies:

uv sync

Configure AWS credentials at ~/.aws/credentials:

[default]aws_access_key_id = AKIA...
aws_secret_access_key = ...

And the default region at ~/.aws/config:

[default]region = eu-central-1

All scripts in this project target eu-central-1. If you use a different region, update the region_name argument (and the AvailabilityZone in the restore script) accordingly.

---

Overview

Create EC2 instances

Launch 2 instances:

• Number of instances:2
• Instance type:t2.micro
• Storage:1 volume — 8 GiB

Tag the instances — name one dev and the other prod:

Confirm each instance has its volume attached:

Write a Python script that automates creating backups for EC2 Volumes

Boto3 docs:

• boto3.client
• describe_volumes
• create_snapshot

importboto3ec2_client=boto3.client('ec2', region_name='eu-central-1')
volumes=ec2_client.describe_volumes()
forvolumeinvolumes['Volumes']:
new_snapshot=ec2_client.create_snapshot(
VolumeId=volume['VolumeId']
)
print(new_snapshot)

Run

python3 volume-backups.py

Check snapshots in AWS Dashboard

Scheduler implementation

importboto3importscheduleimporttimeec2_client=boto3.client('ec2', region_name='eu-central-1')
defcreate_volume_snapshots():
volumes=ec2_client.describe_volumes()
forvolumeinvolumes['Volumes']:
new_snapshot=ec2_client.create_snapshot(
VolumeId=volume['VolumeId']
)
print(new_snapshot)
schedule.every().day.do(create_volume_snapshots)
whileTrue:
schedule.run_pending()
time.sleep(1)

For a quick demo — shorten the interval to a few seconds:

schedule.every(20).seconds.do(create_volume_snapshots)

Backup only Production Servers

First, tag the production volume. Navigate to the prod instance → Storage → select its attached volume:

Set the volume name to prod:

Then filter describe_volumes by that tag so only the prod volume is backed up (Filters reference):

volumes=ec2_client.describe_volumes(
Filters=[
{
'Name': 'tag:Name',
'Values': ['prod']
}
]
)

python3 volume-backups.py

Check the source volume id — it should match the prod volume:

Write a Python script that cleans up old EC2 Volume snapshots

Doc: describe_snapshots

First, list the snapshots you own, sorted newest-first (reverse=True):

importboto3fromoperatorimportitemgetterec2_client=boto3.client('ec2', region_name='eu-central-1')
snapshots=ec2_client.describe_snapshots(
OwnerIds=['self']
)
sorted_by_date=sorted(snapshots['Snapshots'], key=itemgetter('StartTime'), reverse=True)
forsnapinsnapshots['Snapshots']:
print(snap['StartTime'])
print('############')
forsnapinsorted_by_date:
print(snap['StartTime'])

With the list sorted newest-first, slicing [2:] skips the two most recent and deletes everything older:

Boto3 doc: delete_snapshot

importboto3fromoperatorimportitemgetterec2_client=boto3.client('ec2', region_name='eu-central-1')
snapshots=ec2_client.describe_snapshots(
OwnerIds=['self']
)
sorted_by_date=sorted(snapshots['Snapshots'], key=itemgetter('StartTime'), reverse=True)
forsnapinsorted_by_date[2:]:
response=ec2_client.delete_snapshot(
SnapshotId=snap['SnapshotId']
)
print(response)

Run:

python3 cleanup-snapshots.py

Check the AWS dashboard — only the 2 most recent snapshots remain:

Cleanup for multiple volumes

importboto3fromoperatorimportitemgetterec2_client=boto3.client('ec2', region_name='eu-central-1')
volumes=ec2_client.describe_volumes(
Filters=[
{
'Name': 'tag:Name',
'Values': ['prod']
}
]
)
forvolumeinvolumes['Volumes']:
snapshots=ec2_client.describe_snapshots(
OwnerIds=['self'],
Filters=[
{
'Name': 'volume-id',
'Values': [volume['VolumeId']]
}
]
)
sorted_by_date=sorted(snapshots['Snapshots'], key=itemgetter('StartTime'), reverse=True)
forsnapinsorted_by_date[2:]:
response=ec2_client.delete_snapshot(
SnapshotId=snap['SnapshotId']
)
print(response)

Write a Python script that restores EC2 Volumes

Grab the prod instance id from the AWS dashboard:

Boto3 docs:

• boto3.resource
• describe_volumes — filtered by attachment.instance-id
• describe_snapshots
• create_volume
• Volume resource — used to poll vol.state
• Instance resource and its attach_volume action

Get the device id:

A new volume can't reuse a device name already in use, so change the device from /dev/xvda to /dev/xvdb.

Replace instance_id with your own prod instance id, and make sure AvailabilityZone matches the instance's AZ — a volume can only attach to an instance in the same Availability Zone. The script polls vol.state until the new volume is available, then attaches it.

importboto3importtimefromoperatorimportitemgetterec2_client=boto3.client('ec2', region_name='eu-central-1')
ec2_resource=boto3.resource('ec2', region_name='eu-central-1')
instance_id='i-0b62c30dfaaf087c5'device_id='/dev/xvdb'volumes=ec2_client.describe_volumes(
Filters=[
{
'Name': 'attachment.instance-id',
'Values': [instance_id]
}
]
)
instance_volume=volumes['Volumes'][0]
print(instance_volume)
snapshots=ec2_client.describe_snapshots(
OwnerIds=['self'],
Filters=[
{
'Name': 'volume-id',
'Values': [instance_volume['VolumeId']]
}
]
)
latest_snapshot=sorted(snapshots['Snapshots'], key=itemgetter('StartTime'), reverse=True)[0]
print(latest_snapshot['StartTime'])
new_volume=ec2_client.create_volume(
SnapshotId=latest_snapshot['SnapshotId'],
AvailabilityZone='eu-central-1a',
TagSpecifications=[
{
'ResourceType': 'volume',
'Tags': [
{
'Key': 'Name',
'Value': 'prod'
}
]
}
]
)
whileTrue:
vol=ec2_resource.Volume(new_volume['VolumeId'])
print(vol.state)
ifvol.state=='available':
ec2_resource.Instance(instance_id).attach_volume(
VolumeId=new_volume['VolumeId'],
Device=device_id
)
breaktime.sleep(1)

Run:

python3 restore-volume.py

Check the restored volume on AWS — it's attached to the prod instance as a second device:

Clean up

Tear down everything created from the AWS console once you're done:

1. Instances
2. Snapshots
3. Volumes