Skip to content

Repository files navigation

EventLab

EventLab is an interactive distributed-systems failure laboratory. It makes message delivery, retries, dead letters, ordering, saga compensation, replay, and recovery visible through a public web interface and distributed traces.

Its Load & Concurrency Lab launches bounded burst or steady traffic through the real distributed workflow and calls the result proved only when the backlog drains and every accepted member's backend invariant assessment passes.

Open the permanent EventLab project tour

The project is a portfolio and learning system, not a parcel-management product. Its deliberately small order-fulfilment workflow exists only to provide realistic distributed-system experiments.

Status

Milestones 0–30 are complete. EventLab includes a disposable Azure environment, a permanent portfolio tour, repeatable resilience measurements, self-contained curated, custom, and load laboratories with warmed order-balanced concurrency trials, variability reporting, transaction-boundary pipeline diagnostics, first-to-last wave timing, live broker-pressure sampling, automatic stall attribution, controlled bottleneck experiments, guided baseline-versus-constrained impact reports, and shareable load-result evidence, durable Run Inspector evidence, configurable recovery policies, backend-generated downloadable JSON and Markdown reports, a live deployment Control Center, fail-closed evidence-pipeline readiness, authoritative-versus-projected consistency proof, a traceable poison-message quarantine experiment, and non-destructive native DLQ inspection.

Target technology

  • Java 21, Spring Boot 3, and Maven multi-module builds.
  • React, TypeScript, and Vite.
  • PostgreSQL and Flyway.
  • Azure Service Bus Standard in Azure and the official Service Bus emulator locally.
  • Docker and Docker Compose.
  • Azure Container Apps.
  • Terraform.
  • OpenTelemetry, with Grafana/Tempo for self-contained trace demos and Azure Monitor/Application Insights for Azure operations.
  • JUnit 5 and Testcontainers.
  • GitHub Actions, GitHub Container Registry, and GitHub OIDC federation with Microsoft Entra ID.

Documentation

The permanent static tour is published through GitHub Pages from the same frontend with VITE_STATIC_TOUR=true. It contains no API calls and remains useful while the disposable Azure lab is offline. Locally, open http://localhost:5173/?tour after starting Vite.

Guiding rule

A feature belongs in the MVP only if it enables, exposes, verifies, or explains a distributed-systems behavior.

Repository structure

  • contracts/ — transport-neutral event, workflow, and scenario contracts.
  • services/workflow-service/ — future workflow and saga orchestrator.
  • services/payment-service/ — simulated payment participant.
  • services/fulfilment-service/ — simulated fulfilment participant.
  • services/lab-console/ — experiment control plane and timeline projection.
  • frontend/ — React experiment console.
  • infrastructure/ — container and, later, Azure/Terraform assets.
  • docs/ — architecture, roadmap, and decision records.

Verify locally

.\mvnw.cmd verify
Set-Location frontend
npm.cmd install
npm.cmd test
npm.cmd run build
Set-Location ..
docker compose config --quiet

Run the local stack

The local broker is Microsoft's official Azure Service Bus emulator. It depends on an Azure SQL Edge container and requires you to accept Microsoft's container EULA explicitly. Review the applicable terms, then copy the environment template and change ACCEPT_EULA=N to ACCEPT_EULA=Y. EventLab never accepts it automatically.

Copy-Item .env.example .env
# Review .env and explicitly set ACCEPT_EULA=Y if you accept the terms.
docker compose up -d postgres servicebus-sql servicebus tempo otel-collector grafana

The PostgreSQL initialization script creates one database and role per service. If the Postgres volume predates that script, remove only that development volume and recreate it:

docker compose down
docker volume rm eventlab_eventlab-postgres-data
docker compose up -d postgres

Package the multi-module backend once from the repository root:

mvn package

Then start the four backend processes in separate PowerShell terminals from the repository root. Each terminal must enable messaging before launching its service JAR:

$env:EVENTLAB_MESSAGING_ENABLED='true'
java -jar services/workflow-service/target/workflow-service-0.1.0-SNAPSHOT.jar
$env:EVENTLAB_MESSAGING_ENABLED='true'
java -jar services/payment-service/target/payment-service-0.1.0-SNAPSHOT.jar
$env:EVENTLAB_MESSAGING_ENABLED='true'
java -jar services/fulfilment-service/target/fulfilment-service-0.1.0-SNAPSHOT.jar
$env:EVENTLAB_MESSAGING_ENABLED='true'
java -jar services/lab-console/target/lab-console-0.1.0-SNAPSHOT.jar

Then start the UI:

Set-Location frontend
npm.cmd run dev

Open http://localhost:5173, run Successful payment workflow, and follow any trace link into Grafana at http://localhost:3000.

The emulator connection string is deliberately static and local-only. The emulator does not persist broker state across restarts; its queues, topic, and subscriptions are recreated from infrastructure/servicebus/Config.json.

After packaging the backend and starting the Compose infrastructure, the cross-service duplicate-delivery check can be repeated without the UI:

mvn package
.\scripts\verify-duplicate-scenario.ps1
.\scripts\verify-dlq-recovery.ps1
.\scripts\verify-compensation-scenario.ps1
.\scripts\verify-ordering-scenario.ps1
.\scripts\verify-custom-experiment.ps1
.\scripts\verify-automatic-recovery.ps1

Run the invariant-aware concurrent baseline with the official k6 container:

docker compose --profile performance run --rm k6
.\scripts\verify-payment-restart-recovery.ps1
.\scripts\verify-outbox-acknowledgement-window.ps1

Reliability model

Workflow, Payment, and Fulfilment persist business state and outgoing messages in the same database transaction. Scheduled dispatchers send pending outbox rows and mark successful delivery. A crash after the broker accepts a message but before the row is marked can still produce a duplicate—as expected under at-least-once delivery—so consumers claim the logical message ID in an inbox within the same transaction as their state change. Failed Fulfilment attempts deliberately do not claim the inbox entry; the claim occurs only when the command succeeds.

About

Interactive distributed-systems failure laboratory

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages