Microservices Audio Platform

A scalable, containerized microservices architecture for MP3 processing and metadata management.

---

📐 Architecture Overview

Core Services

• Resource Service — MP3 storage & processing (S3 + DB)
• Song Service — Metadata management
• Resource Processor — Async metadata extraction via RabbitMQ
• Gateway — API gateway + routing (Spring Cloud Gateway + Eureka)
• Discovery Service — Eureka service registry (port 8761)
• Config Service — Git-backed Spring Cloud Config Server (port 8888)
• Storage Service — Shared storage operations
• QA Service — Integration & E2E testing

---

🖼️ Key Diagrams

Communication	Fault Tolerance	Containerization	Service Discovery

---

🚀 Quick Start

./gradlew clean build

Microservices Audio Platform

docker compose up -d --build

Run all services locally (correct startup order):

./gradlew :config-service:bootRun
./gradlew :discovery-service:bootRun
./gradlew :gateway:bootRun
./gradlew :auth-service:bootRun
./gradlew :resource-service:bootRun
./gradlew :song-service:bootRun
./gradlew :storage-service:bootRun
./gradlew :resource-processor:bootRun
./gradlew :qa-service:bootRun
./gradlew :ui-service:bootRun

Note:ui-service is a React/Node.js project (not Spring Boot). When you run ./gradlew :ui-service:bootRun, it delegates to the startReactApp task which executes npm start. Alternatively, you can run it directly: cd ui-service && npm start.

Local dev (without Docker for services):

docker compose up -d resource-db song-db storage-db rabbitmq localstack

./gradlew clean build -x test
docker compose up -d config-service discovery-service
docker compose up -d resource-service song-service resource-processor storage-service gateway
curl http://localhost:8761
curl http://localhost:8888/actuator/health curl http://localhost:8080/actuator/health
curl http://localhost:8080/resources/actuator/health
docker compose down -v

Service Discovery stack (Eureka + Gateway + Git Config) is fully implemented and required for all inter-service communication and dynamic routing. All clients use spring.config.import: configserver:... + Eureka registration

---

🧪 Testing

• Unit, Integration, Component (Jbehave), Contract & E2E tests
• Postman collection + sample MP3 files included in tools/
• Allure + JBehave reports generated

---

📚 Documentation

• Introduction & Requirements
• Fundamentals
• Communication & Messaging
• Containerization
• Testing Strategy
• Fault Tolerance
• Storage State Machine

---

📁 Project Structure

Microservices/
├── resource-service/
├── song-service/
├── resource-processor/
├── gateway/ # + reactive routes, Error handler, discovery.locator
├── discovery-service/ # Eureka @EnableEurekaServer
├── config-service/ # Git-backed Config Server
├── storage-service/
├── qa-service/
├── config-repo/ # Per-service + docker ymls (authoritative)
├── compose.yaml
├── .env
├── tools/
│ ├── images/
│ ├── docs/
│ └── api-tests/
└── README.md

---

✨ Features

• ✅ Async processing with RabbitMQ + retries
• ✅ Cloud storage (LocalStack S3)
• ✅ PostgreSQL per service (Alpine)
• ✅ Two-stage Docker builds
• ✅ Health checks & proper startup ordering
• ✅ Global error handling & validation (Gateway: structured JSON via WebExceptionHandler)
• ✅ Service Discovery (Eureka + Spring Cloud Gateway + Git Config Server + @RefreshScope)
• ✅ Full API test coverage
• ✅ Fault tolerance: Resilience4j Circuit Breaker + Spring Retry, STAGING→PERMANENT state machine, configurable stub fallback, S3 health probe

---

🚨 Module 7: Observability Pipeline (ELK + Tracing)

Stack

Elasticsearch + Logstash + Kibana | Micrometer Tracing

What was implemented

Centralized Logging (ELK Stack):

• ✅ Elasticsearch 7.17.10 — log storage & indexing (logs-* indices)
• ✅ Logstash 7.17.10 — TCP/UDP input (port 5000), JSON parsing, Elasticsearch output
• ✅ Kibana 7.17.10 — log visualization & trace ID search (http://localhost:5601)
• ✅ JSON-formatted logs via logstash-logback-encoder (severity, service, traceId, message)
• ✅ logback-spring.xml in all 5 services (gateway, resource-service, song-service, resource-processor, storage-service)
• ✅ json-file Docker logging driver (max-size: 10m, max-file: 3)

Distributed Tracing:

• ✅ TraceGatewayFilter — Injects/generates X-Trace-Id at API Gateway ingress
• ✅ TraceIdInterceptor — Extracts trace ID from HTTP headers in resource-service & song-service
• ✅ WebClientTraceConfig — Propagates trace ID to downstream HTTP calls via ExchangeFilterFunction
• ✅ RabbitMQ trace propagation — X-Trace-Id header sent with messages, extracted by ResourceProcessor
• ✅ Sleuth/Micrometer Tracing config in config-repo/application.yml (W3C propagation, 100% sampling)
• ✅ All logs correlated by traceId across Gateway → Resource Service → Storage Service → Song Service → Resource Processor

Architecture Flow:

[Client] → Gateway (injects X-Trace-Id)
├─ Resource Service (intercepts traceId, logs JSON, propagates via WebClient + RabbitMQ)
│ ├─ Storage Service (logs with traceId)
│ └─ Song Service (logs with traceId)
└─ RabbitMQ → Resource Processor (receives traceId, logs with traceId)
[All services emit JSON logs → Logstash → Elasticsearch → Kibana (:5601)]

---

Containerization Implementation Review

Dockerfiles

• Two-stage builds: All 7 Dockerfiles use two-stage builds.
• WORKDIR /app: All Dockerfiles correctly set /app as the working directory.
• COPY for file transfers: All use COPY (not ADD) for local files.
• Wildcard JAR files: COPY --from=builder /app/*/build/libs/*.jar app.jar.
• Alpine runtime base: eclipse-temurin:21-jre-alpine.
• Alpine build base: gradle:8.8-jdk21-alpine.
• CMD used: All 7 Dockerfiles use CMD ["java", "-jar", "app.jar"].
• EXPOSE correct ports per service: 8888, 8761, 8080, 8081, 8082, 8083, 8085.
• Dependency caching: Gradle configs copied before source, ./gradlew :service:dependencies runs first.
• ./gradlew used: All use the Gradle Wrapper.
• assemble --no-daemon -x test: Faster builds without tests.
• Alpine apk for curl: apk add --no-cache curl (lightweight).

Docker Compose (compose.yaml)

• PostgreSQL 17-alpine: All DBs use postgres:17-alpine.
• Health checks on all services.
• depends_on with condition: service_healthy.
• Environment variables from .env.
• RabbitMQ + LocalStack configured properly.
• build used (not image) for microservice containers.
• Single command deployment: docker compose up -d --build.
• Service name resolution: Logical names used.
• Default network (no custom microservices-net).
• No named volumes for DB persistence.
• init-scripts/ directory mounted to /docker-entrypoint-initdb.d.

Application Configuration

• Spring Config Server integration: spring.config.import: configserver:....
• Actuator health endpoints exposed.
• Env var fallbacks: ${VAR:default} pattern used everywhere.
• Eureka URLs: ${EUREKA_URL:http://localhost:8761/eureka/}.
• Config server URLs: ${CONFIG_SERVER_URL:http://localhost:8888}.
• RabbitMQ config: env var fallbacks for host/port/user/password.
• Resource-processor ports/URLs: env var fallbacks.
• storage-service application.yaml: created with env var fallbacks.

Infrastructure

• .env file present.
• .dockerignore files present for most services.
• init-scripts/ directory: resource-db/init.sql, song-db/init.sql, storage-db/init.sql.
• SQL scripts only define tables (no CREATE DATABASE).

---

Allure

http://localhost:9200/_cat/indices?vhttp://localhost:5601/app/management/kibana/dataViewshttp://localhost:9200/_cluster/healthhttp://localhost:5601/api/statushttp://localhost:4566/_localstack/healthhttp://localhost:15672/http://localhost:9090/-/healthyhttp://localhost:3090/api/healthhttp://localhost:9600/http://localhost:9000/auth/oauth2/token