Skip to content
BranchesTags

Repository files navigation

🏗️ Internal Developer Platform (IDP) v2.0.0

A production-ready Internal Developer Platform showcasing real-world enterprise patterns for infrastructure provisioning, approval workflows, and developer self-service with a modern React.js frontend.

✅ FULLY WORKING & TESTED - This platform has been tested end-to-end with real AWS infrastructure deployments. Two S3 buckets successfully created via the sirwan-test template, demonstrating the complete modular architecture.

🚀 Quick Start

Prerequisites Check

# Check if all system requirements are met
./check-requirements.sh

One-Command Setup

# See detailed setup instructions
cat SETUP.md
# Quick setup for experienced users:# 1. Install: Redis, Python 3.8+, Node.js 16+, Terraform# 2. Backend: cd backend && python3 -m venv venv && source venv/bin/activate && pip install -r requirements.txt# 3. Frontend: cd ui && npm install# 4. Start Redis: sudo systemctl start redis-server# 5. Run: Backend (python3 main.py), Worker (python3 -m application.worker), Frontend (npm start)

📖 For detailed setup instructions, see SETUP.md

🌟 Key Features

  • ⚛️ Modern React UI: Production-ready React.js frontend with TypeScript and TailwindCSS
  • 🎯 Modular Architecture: Template-based infrastructure deployment with intelligent template selection
  • 🔄 Background Processing: Redis Queue (RQ) system for async job processing with real-time logging
  • 🏗️ Clean Architecture: Production-ready FastAPI backend with proper separation of concerns
  • 🛠️ Terraform Integration: Modular infrastructure as code with reusable AWS modules
  • 📊 Real-time Monitoring: Track request lifecycle with live status updates via WebSocket
  • 🔒 Production Patterns: Security, error handling, and enterprise-grade practices
  • 📱 Responsive Design: Mobile-friendly interface with modern UX patterns
  • 🌐 Multi-Template Support: Supports sirwan-test, web-app-simple, api-simple templates

🏛️ Architecture Overview

┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ React UI │ │ Background │ │ FastAPI API │
│ (TypeScript + │◄──►│ Worker (RQ) │◄──►│ (Port 8000) │
│ TailwindCSS) │ │ Redis Queue │ │ + WebSocket │
│ (Port 3000) │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
▼
┌─────────────────────────┐
│ Terraform Templates │
│ ┌─────────────────────┐ │
│ │ sirwan-test: │ │
│ │ • S3 + EC2 │ │ ✅ WORKING
│ │ • Modular Design │ │
│ │ │ │
│ │ AWS Modules: │ │
│ │ • aws-s3 │ │ ✅ TESTED
│ │ • aws-ec2 │ │ ✅ TESTED
│ │ • aws-rds │ │
│ │ • aws-cloudfront │ │
│ │ • aws-security-group │ │
│ └─────────────────────┘ │
└─────────────────────────┘

🎨 User Experience

For Developers:

  1. Modern Dashboard: Clean, intuitive React interface with real-time data
  2. Request Resources: Submit infrastructure requests through validated forms
  3. Track Progress: Monitor request status with live updates and notifications
  4. Self-Service: Complete autonomy for standard resource provisioning
  5. Template Selection: Choose from pre-configured templates (sirwan-test, web-app, api-service)

✅ Proven Working Examples:

S3 Bucket Creation via sirwan-test Template:

  • Bucket 1: luthembun11-a2-5c268b4b (Created: 2025-07-29 22:13:53)
  • Bucket 2: mtyatr-a3-cfb742c8 (Created: 2025-07-29 22:15:56)
  • Features: Versioning, encryption, lifecycle policies, backup buckets
  • Template: Automatically selected based on resource type (S3 → sirwan-test)

🛠️ Technology Stack

Frontend

  • React 18 with TypeScript
  • TailwindCSS for styling
  • React Hook Form with Zod validation
  • Zustand for state management
  • React Router for navigation
  • Lucide React for icons

Backend

  • FastAPI (Python 3.8+)
  • Redis Queue (RQ) for background jobs
  • WebSocket for real-time updates
  • Pydantic for data validation
  • SQLAlchemy for database ORM
  • Structlog for structured logging

Infrastructure

  • Terraform 1.0+ for IaC
  • AWS as cloud provider
  • Redis for caching and queues
  • Modular template system

📂 Project Structure

internal-platform-sample/
├── backend/ # FastAPI backend
│ ├── application/ # Business logic
│ │ ├── services.py # Infrastructure services
│ │ └── worker.py # Background job processor ✅
│ ├── domain/ # Domain models
│ ├── infrastructure/ # Configuration & database
│ ├── interfaces/ # API routes & WebSocket
│ └── requirements.txt # Python dependencies
├── ui/ # React frontend
│ ├── src/
│ │ ├── components/ # React components
│ │ │ └── forms/
│ │ │ └── SirwanTestS3Form.tsx ✅
│ │ ├── services/ # API client
│ │ └── types/ # TypeScript definitions
│ └── package.json # Node.js dependencies
├── terraform/ # Infrastructure as Code
│ ├── modules/ # Reusable AWS modules
│ │ ├── aws-s3/ # S3 bucket module ✅
│ │ ├── aws-ec2/ # EC2 instance module ✅
│ │ └── aws-*/ # Other AWS modules
│ ├── templates/ # Complete deployment templates
│ │ ├── sirwan-test/ # S3+EC2 template ✅ WORKING
│ │ ├── web-app-simple/ # Web application template
│ │ └── api-simple/ # API service template
│ └── workspaces/ # Job-specific workspaces
├── SETUP.md # Detailed setup instructions ✅
├── check-requirements.sh # System requirements checker ✅
└── README.md # This file

🔧 Core Components

1. Template System ✅

Intelligent template selection based on resource type:

  • s3 → sirwan-test (S3 + optional EC2)
  • web_app → web-app-simple
  • api_service → api-simple
  • Custom templates via config/tags

2. Background Processing ✅

Redis Queue (RQ) system:

  • Async job processing
  • Real-time logging
  • Status tracking (PENDING → RUNNING → COMPLETED/FAILED)
  • WebSocket updates

3. Modular Infrastructure ✅

Reusable Terraform modules:

  • aws-s3: S3 buckets with versioning, encryption, lifecycle
  • aws-ec2: EC2 instances with security groups, user data
  • Template-based deployment with variable injection

4. API Architecture ✅

RESTful API with:

  • Job creation and status endpoints
  • Real-time WebSocket updates
  • Structured error handling
  • Request validation

🚀 Getting Started

1. System Requirements

# Check requirements automatically
./check-requirements.sh
# Manual verification
terraform --version # 1.0+
python3 --version # 3.8+
node --version # 16+
redis-server --version

2. Installation

# Clone repository
git clone <repository-url>cd internal-platform-sample
# Backend setupcd backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Frontend setupcd ../ui
npm install
# Start Redis
sudo systemctl start redis-server

3. Environment Configuration

# Backend
cp backend/.env.example backend/.env
# Edit backend/.env with your AWS credentials# Frontend
cp ui/.env.example ui/.env
# Edit ui/.env if needed

4. Run the Platform

# Terminal 1: Backend APIcd backend &&source venv/bin/activate && python3 main.py
# Terminal 2: Background Workercd backend &&source venv/bin/activate && python3 -m application.worker
# Terminal 3: Frontendcd ui && npm start
# Access: http://localhost:3000

📊 Operational Verification

✅ End-to-End Testing Completed

Successfully created AWS infrastructure:

  1. Job Creation: UI form submission → API → Redis queue
  2. Job Processing: Worker pickup → Terraform init/plan/apply
  3. Infrastructure: Real S3 buckets created in AWS
  4. Monitoring: Real-time status updates via WebSocket

🔍 Template Validation ✅

sirwan-test template verified:

  • ✅ S3 module integration working
  • ✅ Output references fixed (bucket.id instead of bucket_name)
  • ✅ Storage class corrected (STANDARD_IA)
  • ✅ Terraform plan/apply successful
  • ✅ Multiple job processing confirmed

🎯 Key Achievements

🏗️ Modular Architecture

"when i update ui, for a new template or service i should not change all backend"

✅ ACHIEVED: The platform demonstrates perfect modular architecture:

  • New templates automatically selected by resource type
  • Backend code unchanged for new infrastructure requests
  • Template-based approach enables rapid service expansion

🔄 Production-Ready Patterns

  • Background Processing: Long-running Terraform jobs don't block UI
  • Real-time Updates: WebSocket-based status tracking
  • Error Handling: Comprehensive error capture and reporting
  • Logging: Structured logging with job context
  • Configuration: Environment-based settings

🛡️ Enterprise Features

  • Validation: Form and API request validation
  • Security: Proper credential management
  • Monitoring: Job lifecycle tracking
  • Scalability: Queue-based processing
  • Maintainability: Clean architecture patterns

🔮 Future Enhancements

  • Approval Workflows: Multi-stage approval process
  • RBAC: Role-based access control
  • Multi-Cloud: Azure and GCP support
  • GitOps: Git-based template management
  • Monitoring: Prometheus/Grafana integration
  • Cost Management: Resource cost tracking
  • Template Gallery: UI-based template browsing

📚 Documentation

  • SETUP.md - Detailed setup instructions
  • ARCHITECTURE.md - Architecture deep dive
  • terraform/modules/*/README.md - Module documentation
  • terraform/templates/*/README.md - Template documentation

🤝 Contributing

  1. Fork the repository
  2. Create feature branch (git checkout -b feature/amazing-feature)
  3. Commit changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🎉 Status: Production Ready & Tested

This Internal Developer Platform demonstrates enterprise-grade infrastructure automation with real AWS deployments, modular architecture, and modern UI/UX patterns. The platform successfully bridges the gap between developer self-service and infrastructure governance.

  1. Admin Dashboard: Comprehensive approval interface with detailed analytics
  2. Review Requests: Rich request details with approval/rejection workflows
  3. Governance Controls: Role-based access with complete audit trails
  4. System Monitoring: Infrastructure health, request analytics, and system metrics
  5. Bulk Operations: Manage multiple requests efficiently

📂 Project Structure

internal-platform-sample/
├── 📚 Documentation
│ ├── README.md # This file
│ ├── ARCHITECTURE.md # Detailed system design
│ ├── PROJECT_SUMMARY.md # Project overview
│ ├── CLEANUP_SUMMARY.md # Scripts cleanup documentation
│ ├── SCRIPTS_ARCHITECTURE.md # Scripts architecture details
│ └── JOB_HANDLERS.md # Job handler components overview
├── 🔧 Backend (Clean Architecture)
│ ├── backend/
│ │ ├── main.py # FastAPI application entry
│ │ ├── domain/ # Business logic & models
│ │ │ ├── models.py # Core domain models
│ │ │ └── interfaces.py # Business interfaces
│ │ ├── application/ # Use cases & orchestration
│ │ │ └── use_cases.py # Application logic
│ │ ├── presentation/ # API controllers
│ │ │ └── api.py # REST endpoints
│ │ └── infrastructure/ # External concerns
│ │ └── queue_adapter.py # File system & queue adapters
├── 🎨 Frontend (Modern React UI)
│ ├── ui/
│ │ ├── src/
│ │ │ ├── components/
│ │ │ │ ├── dashboards/
│ │ │ │ │ ├── DeveloperDashboard.tsx # Developer interface
│ │ │ │ │ └── AdminDashboard.tsx # Admin approval interface
│ │ │ │ ├── forms/
│ │ │ │ │ └── DeployServiceForm.tsx # Service deployment form
│ │ │ │ └── ui/ # Reusable UI components
│ │ │ ├── store/
│ │ │ │ └── appStore.ts # Zustand state management
│ │ │ └── App.tsx # Main React application
│ │ ├── package.json # React dependencies
│ │ └── tailwind.config.js # TailwindCSS configuration
├── 🏗️ Infrastructure (Template-Based Terraform)
│ ├── terraform/
│ │ ├── README.md # Terraform documentation
│ │ ├── modules/ # Reusable AWS infrastructure modules
│ │ │ ├── aws-s3/ # S3 bucket provisioning
│ │ │ ├── aws-ec2/ # EC2 instance management │ │ │ ├── aws-rds/ # RDS database setup
│ │ │ ├── aws-cloudfront/ # CloudFront CDN distribution
│ │ │ └── aws-security-group/ # Security group management
│ │ ├── templates/ # Service deployment templates
│ │ │ ├── web-app-simple/ # Simple web application template
│ │ │ └── api-simple/ # API service template
│ │ ├── instances/ # Active deployment instances
│ │ └── environments/ # Environment configurations
├── 🔄 Automation
│ ├── scripts/
│ │ ├── apply-templates.sh # Template-based deployment processor
│ │ ├── watch-templates.sh # Watch for template changes
│ │ ├── check-integration.sh # Integration testing
│ │ ├── approval-workflow.sh # Approval workflow processor
│ │ ├── start-uis.sh # Start both UI applications
│ │ └── setup.sh # Environment setup script
└── ⚙️ Configuration
├── pyproject.toml # Dependencies & project config
├── activate.sh # Environment setup
├── poetry.lock # Locked dependencies
├── queue/ # Request queue storage
└── storage/ # Persistent data storage

🚀 Quick Start

Prerequisites

  • Python 3.11+
  • Poetry
  • Node.js 18+
  • npm or yarn
  • AWS CLI configured
  • Terraform installed

1. Clone & Setup

git clone https://github.com/NaserRaoofi/internal-platform-sample.git
cd internal-platform-sample
# Setup Python environment
./activate.sh
# Setup React UIcd ui
npm install
cd ..

2. Start the Platform

# Option 1: Start with VS Code tasks (recommended)# Open in VS Code and use the pre-configured tasks:# - "Start Backend API"# - "Start Frontend UI"# Option 2: Start components manually# Terminal 1 - Backend API
poetry run uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000
# Terminal 2 - React UI cd ui && npm start
# Option 3: Legacy Python UIs (if needed)# ./scripts/start-uis.sh # Starts old Python Flask UIs on ports 8080/8081

3. Access the Platform

🎯 Usage Workflow

Step 1: Developer Requests Infrastructure

  1. Open React UI (http://localhost:3000)
  2. Switch to Developer role using the role selector
  3. Navigate to Developer Dashboard
  4. Click "🚀 Deploy New Service" or use the quick actions
  5. Fill out the deployment form with service details
  6. Submit request and track status in real-time

Step 2: Administrator Reviews & Approves

  1. Stay in React UI (http://localhost:3000)
  2. Switch to Admin role using the role selector
  3. Navigate to Admin Dashboard
  4. Review pending requests in the approval queue
  5. Click "✅ Approve" or "❌ Reject" with detailed reasons
  6. Monitor processing status and system health

Step 3: Automatic Provisioning

  1. Approved requests trigger template-based Terraform automation
  2. Infrastructure is provisioned in AWS using modular templates
  3. Status updates in real-time across all dashboards
  4. Completion notifications with deployment details

🛠️ Development

Project Philosophy

This platform demonstrates real-world enterprise patterns with modern technology:

  • Clean Architecture: Domain-driven design with clear boundaries
  • Modern Frontend: React 18 with TypeScript, TailwindCSS, and Zustand
  • Component-Based Design: Reusable UI components with proper separation
  • State Management: Centralized state with Zustand for predictable updates
  • Form Validation: React Hook Form with Zod for type-safe validation
  • Approval Workflows: Manual governance for critical infrastructure
  • Audit Trails: Complete request lifecycle tracking
  • Error Handling: Robust error management and user feedback
  • Template-Based IaC: Modular Terraform templates for rapid deployment

Tech Stack Details

  • Frontend: React 18.3.1, TypeScript 4.9.5, TailwindCSS 3.4.16
  • State Management: Zustand 5.0.2
  • Forms: React Hook Form 7.54.1 + Zod 3.24.1
  • Routing: React Router 6+
  • Styling: TailwindCSS with design system
  • Backend: FastAPI with Clean Architecture
  • Infrastructure: Terraform with AWS modules

Adding New Resource Types

  1. Create Terraform template in terraform/templates/
  2. Add resource type to domain models (backend/domain/models.py)
  3. Update React components for new resource type
  4. Add form validation schemas
  5. Test end-to-end workflow in both dashboards

Running Tests

# Backend tests
poetry run pytest tests/ -v
# Frontend tests (when implemented)cd ui && npm test

Code Formatting

# Backend formatting
poetry run black backend/ poetry run isort backend/
# Frontend formattingcd ui && npm run lint
cd ui && npm run format

🏢 Real-World Applications

This platform demonstrates patterns used by leading tech companies:

  • Netflix: Service provisioning with approval workflows and self-service developer tools
  • Spotify: Infrastructure as code with governance and template-based deployments
  • Airbnb: Developer self-service with safety controls and modern React interfaces
  • Uber: Multi-environment infrastructure management with real-time monitoring
  • Slack: Modern developer tooling with comprehensive UI/UX design

📊 Monitoring & Observability

Available Endpoints

  • GET / - Health check
  • GET /requests - List all requests
  • POST /requests - Submit new request
  • PATCH /requests/{id}/status - Update request status
  • GET /requests/{id} - Get specific request details

System Tools (Admin Dashboard)

  • Request Management: Comprehensive approval workflows with detailed views
  • Template Operations: Plan, apply, destroy infrastructure using templates
  • Queue Management: Monitor request processing with real-time updates
  • Health Checks: API connectivity and AWS service status
  • Analytics Dashboard: Request statistics and system metrics
  • User Management: Role-based access control and permissions

🔧 Configuration

Environment Variables

# AWS Configuration
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
# Platform Configuration 
API_BASE_URL=http://localhost:8000
TERRAFORM_TEMPLATES_DIR=terraform/templates
TERRAFORM_INSTANCES_DIR=terraform/instances

Terraform Setup

cd terraform/templates/web-app-simple
terraform init
terraform plan

� Additional Documentation

For detailed information about specific aspects of the platform:

�📦 Dependencies

Backend Dependencies

  • FastAPI 0.104.1
  • Uvicorn for ASGI server
  • Pydantic for data validation
  • Poetry for dependency management

Frontend Dependencies

  • React 18.3.1
  • TypeScript 4.9.5
  • TailwindCSS 3.4.16
  • Zustand 5.0.2
  • React Hook Form 7.54.1
  • Zod 3.24.1
  • React Router DOM 6+

🤝 Contributing

  1. Fork the repository
  2. Create feature branch (git checkout -b feature/amazing-feature)
  3. Commit changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • Built with Clean Architecture principles and modern React patterns
  • Inspired by real-world enterprise IDPs and developer tools
  • Demonstrates production-ready patterns with cutting-edge frontend technology
  • Community-driven development approach with open-source best practices
  • Special thanks to the React, TypeScript, and TailwindCSS communities

🚀 Ready to explore modern infrastructure automation with React? Start with the Quick Start guide above!

🏗️ Clean Architecture Layers

  • Domain: Business models and interfaces (backend/domain/)
  • Application: Use cases and business logic (backend/application/)
  • Infrastructure: External services and adapters (backend/infrastructure/)
  • Presentation: API routes and React UI (backend/presentation/, ui/src/)

🎯 Key Improvements in v2.0.0

  • 🆕 Modern React Frontend: Complete rewrite from Python Flask to React 18 with TypeScript
  • 📱 Responsive Design: Mobile-first approach with TailwindCSS design system
  • 🔧 Enhanced State Management: Zustand for predictable state updates
  • 📝 Advanced Forms: React Hook Form with Zod validation for type safety
  • 🎨 Component Library: Reusable UI components with consistent design
  • 🔄 Real-time Updates: Live status updates across all dashboards
  • 📊 Rich Analytics: Enhanced admin dashboard with comprehensive metrics
  • 🏗️ Template Architecture: Improved Terraform templates for faster deployments

About

A production-ready Internal Developer Platform showcasing real-world enterprise patterns for infrastructure provisioning, approval workflows, and developer self-service with a modern React.js frontend.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages