Event-Driven Architecture: DynamoDB $\rightarrow$ Lambda (Terraform)

---

Overview

This repository implements a production-grade, asynchronous, serverless event pipeline on AWS using Terraform. It ingests transactional data via Amazon DynamoDB and dynamically triggers a decoupled AWS Lambda compute step via DynamoDB Streams. This decoupling pattern isolates data absorption from the processing logic, ensuring high throughput, operational resilience, and zero resource starvation on downstream interfaces.

---

Architecture Diagram

flowchart LR
A[Data Ingestion / API] --> | DynamoDB PutItem | B[(Amazon DynamoDB Table)]
B --> | Captures State Changes | C[DynamoDB Streams]
C --> | Asynchronous Batch Trigger | D[AWS Lambda Function]
D --> | Structured Logging | E[Amazon CloudWatch Logs]
style B fill:#4053D6,stroke:#fff,stroke-width:2px,color:#fff
style D fill:#FF9900,stroke:#fff,stroke-width:2px,color:#fff

Loading

Architectural Pillars & Core Patterns

• Asynchronous Micro-Batching (Scalability): Rather than tying compute resources to synchronous client requests, data is offloaded to a sequential stream layer. This protects processing units from horizontal scaling bottlenecks during traffic spikes.
• Least-Privilege Identity Isolation (Security): The AWS Lambda execution context is bound to a custom IAM role explicitly locked down to read stream checkpoints and write log streams, adhering strictly to the principle of zero-trust least privilege.
• On-Demand Infrastructure (Cost Optimization): The DynamoDB layer is provisioned in PAY_PER_REQUEST execution mode, coupled with zero-idle serverless Lambda functions. Operational costs scale linearly with usage-collapsing to absolute zero during idle periods.

---

Technologies Used

• Amazon DynamoDB: High-performance, schema-agnostic NoSQL storage layer.
• DynamoDB Streams: Append-only transaction log stream emitting NEW_IMAGE state modification vectors.
• AWS Lambda: Serverless Python 3.9 compute handler executing isolated processing events.
• Amazon CloudWatch: Real-time logging framework providing audit controls and telemetry.
• Terraform (IaC): Explicit declarative blueprint mapping cloud resources and access planes.

Project Structure

terraform/
├── main.tf # Core AWS Resource Orchestration & Event Source Mappings
├── terraform.tf # Provider Locks & State Locking Configurations
└── lambda_function.py # Python-based Asynchronous Stream Event Handler

Example Ingestion Payload (DynamoDB Stream Event)

{
"Records": [
{
"eventName": "INSERT",
"eventSource": "aws:dynamodb",
"dynamodb": {
"NewImage": {
"TransactionID": {
"S": "TX-99881-A"
},
"Amount": {
"N": "1450.75"
}
}
}
}
]
}

Lambda Handler Implementation

importjsondeflambda_handler(event, context):
print("Initializing Stream Processing Context...")
print(json.dumps(event, indent=2))
forrecordinevent["Records"]:
ifrecord["eventName"] =="INSERT":
new_image=record["dynamodb"]["NewImage"]
tx_id=new_image["TransactionID"]["S"]
amount=new_image["Amount"]["N"]
print(f"EVENT DETECTED → Processing Financial Event: {tx_id} | Value: ${amount}")
return {"statusCode": 200, "message": "Stream event batch processed successfully."}

Orchestration & Lifecycle Commands

Infrastructure Creation

terraform init
terraform plan
terraform apply -auto-approve

Infrastructure Demolition

terraform destroy -auto-approve

Production Hardening Roadmap

• Integrate an Amazon SQS Dead Letter Queue (DLQ) to intercept and capture un-parseable edge cases safely.
• Layer an Amazon API Gateway instance at the ingestion boundary to implement traffic throttling and edge API key validation.
• Inject custom JSON log structured formatters inside the Lambda runtime to feed into analytical cloud dashboards.