SuperCala

High-Assurance Concurrency Fuzzer & Limit-Testing Testbed for Double-Entry Financial Ledgers

A lightweight, standalone Rust testbed inspired by GaloyMoney/cala and Chapter 16 of Dimitri Fontaine's The Art of PostgreSQL. Designed to test, prove, and falsify row-level locking invariants, transactional outbox persistence, and decorrelated jitter backoffs under extreme parallel merchant rush workloads.

---

⚡ TL;DR

• What It Is: A 100% self-contained async Rust testbed (tokio, sqlx, PostgreSQL 16) designed to stress-test high-concurrency double-entry ledger mechanics.
• The Problem: High-concurrency merchant settlement rushes cause lock convoys, connection pool exhaustion, and circular deadlocks in financial ledgers.
• The Benchmark Proof:
  • Sorted Locks (ORDER BY account_id ASC): 258 TPS, 0 deadlocks (40P01), 0 retries across 2,000 txs.
  • Negative Control (Random Arrival Locks): 12.7 TPS, 8,122 deadlocks detected, 8,128 retries.
• Key Invariants: Canonical lock normalization converts lock graphs into DAGs ($O(1)$ deadlocks); decorrelated jitter backoff absorbs packet drop/latency without pool collapse.
• Run in 10 Seconds: docker run -d --name supercala-postgres -p 5433:5432 postgres:16-alpine && cargo run --release -- --workers 100 --tx-per-worker 20 --jitter

---

🤖 Instructions for AI Coding Agents (Cursor / Copilot / Gemini / Claude)

This repository includes a machine-optimized context payload:

• Dense Specification: See llms.txt for machine-readable invariants, schema mappings, and execution flags.

To run this testbed autonomously:

# 1. Start the tuned PostgreSQL 16 testbed
docker run -d --name supercala-postgres --cap-add=NET_ADMIN \
-e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=supercala \
-p 5433:5432 postgres:16-alpine \
-c deadlock_timeout=50ms \
-c autovacuum_vacuum_scale_factor=0.05
# 2. Apply the exact Cala schema migrations
docker exec -i supercala-postgres psql -U postgres -d supercala < migrations/0001_initial_cala_schema.sql
# 3. Run the baseline concurrency proof (100 workers / 2,000 transactions)
cargo run --release -- --database-url "postgres://postgres:postgres@localhost:5433/supercala" --workers 100 --tx-per-worker 20 --jitter

---

🔬 Empirical Proofs & Benchmark Findings

Workload Configuration	Workers	Conns	TPS	Runtime	Deadlocks (40P01)	Retries	Balance Conservation
Sorted Locks + Jitter (Canonical)	100	50	257.8	7.75s	0	0	100% Equal
Sorted Locks + 300 Workers (Saturation)	300	50	174.5	34.39s	0	0	100% Equal
Random Arrival Locks (Chaos Falsification)	100	50	12.7	157.31s	8,122	8,128	100% Equal
Network Degradation (50ms + 1% loss)	50	50	2.46	202.95s	0	0	100% Equal

1. Invariant Verification (Sorted Row Locks)

• Command: cargo run --release -- --workers 100 --tx-per-worker 20 --jitter
• Result: 258 TPS, 7.75s runtime, 0 deadlocks (40P01), 0 retries, 100% mathematical balance equality verified.

2. Negative Control / Chaos Falsification (Random Arrival Locks)

• Command: cargo run --release -- --workers 100 --tx-per-worker 20 --jitter --chaos-random-locks
• Result: 12.71 TPS, 157.31s runtime, 8,122 deadlocks detected, 8,128 retries.
• Takeaway: Mathematically proves that sorting account IDs lexicographically (ORDER BY account_id ASC FOR UPDATE) transforms the lock acquisition graph into a DAG, eliminating 100% of deadlock cycles.

3. Network Chaos Test (tc netem 50ms Latency + 1% Packet Loss)

• Command:

  docker exec -u 0 supercala-postgres tc qdisc add dev eth0 root netem delay 50ms 10ms loss 1%
  cargo run --release -- --workers 50 --tx-per-worker 10 --jitter
  docker exec -u 0 supercala-postgres tc qdisc del dev eth0 root

• Result: 2.46 TPS, 202.95s runtime, 0 deadlocks, 0 retries. Demonstrates how network latency inflates row-lock residency from 0.5ms to ~400ms across 6 SQL wire round-trips per transaction.

---

🏛️ Architecture & Enterprise Specifications

• Decoupled Sovereign Key Custody & Zero-Trust Transport: Master architectural whitepaper detailing 4-tier decoupled on-premise hardware signing enclaves (reverse gRPC pull streams, zero inbound firewall ports, BIP-340 synthetic nonces).
• Architecture Invariant Specification: Technical breakdown of row-level lock ordering DAGs, outbox event atomicity, and pool starvation defenses.

---

📜 Security & License

• Security Policy: See SECURITY.md for private vulnerability reporting.
• Contributing: See CONTRIBUTING.md for pull request guidelines.
• License: Dual-licensed under MIT / Apache-2.0. Built by @bootlace-dev.