Kuber — A Distributed Knowledge Delivery System for Machine Learning at Scale

KUber is a proof-of-concept system in which multiple independent learning entities train deep neural networks on heterogeneous (non-i.i.d.) local datasets and voluntarily share discovered knowledge with one another through a central metadata registry and a model vault.

---

Repository layout

Kuber_clean_code/
├── kuber_base/ Four learning entities (DNN training + KD)
├── kuber_service/ Metadatabase REST API (entity & knowledge registry)
├── kuber_vault/ Knowledge Vault REST API (model file storage)
└── requirements.txt

Each sub-project has its own README.md with detailed documentation.

---

Architecture overview

┌──────────────────────────────────────────────────────────────────┐
│ kuber_base │
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
│ │ entity_0 │ │ entity_1 │ │ entity_2 │ │ entity_3 │ │
│ │ (non-IID) │ │ (non-IID) │ │ (non-IID) │ │(non-IID) │ │
│ └─────┬──────┘ └─────┬──────┘ └─────┬──────┘ └────┬─────┘ │
│ │ │ │ │ │
└────────┼───────────────┼───────────────┼───────────────┼────────┘
│ register / query knowledge │
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ kuber_service │ │ kuber_vault │
│ (port 5001) │◄─────►│ (port 5002) │
│ │ index │ │
│ SQLite metabase: │ │ Filesystem store: │
│ • entities table │ │ • <kid>.pt files │
│ • knowledge table │ │ │
└─────────────────────┘ └─────────────────────┘

Learning loop per entity (round by round)

for each round:
1. Train locally on private non-i.i.d. data (SGD, LOCAL_EPOCHS)
2. Evaluate on shared common test set
if accuracy improved → Case 1: SHARE
• Upload model state-dict to kuber_vault
• Register knowledge metadata (accuracy, vault_path) at kuber_service
if accuracy degraded → Case 2: RETRIEVE
• Query kuber_service: find knowledge with higher accuracy
• Download teacher model from kuber_vault
• Apply logit-based knowledge distillation to local model
• Re-evaluate

---

Quick start

1. Install dependencies

pip install -r requirements.txt

2. Start kuber_service (terminal 1)

cd kuber_service
python app.py
# Listening on http://localhost:5001

3. Start kuber_vault (terminal 2)

cd kuber_vault
python app.py
# Listening on http://localhost:5002

4. Run all four learning entities (terminal 3)

cd kuber_base
python run_all.py

Or run a single entity:

cd kuber_base
python run_entity.py 0 # entity_id ∈ {0, 1, 2, 3}

---

Configuration

All tunable parameters are in kuber_base/config.py:

Parameter	Default	Description
DIRICHLET_ALPHA	0.5	Non-i.i.d. degree (lower → more heterogeneous)
NUM_ROUNDS	20	Training rounds per entity
LOCAL_EPOCHS	5	SGD epochs per round
LEARNING_RATE	0.01	Base SGD learning rate
COMMON_TEST_SIZE	2000	Shared test samples for fair comparison
KD_TEMPERATURE	4.0	Softmax temperature for knowledge distillation
KD_ALPHA	0.5	CE weight in KD loss (1−α = KD weight)
KD_EPOCHS	3	Fine-tuning epochs during distillation

---

Dependencies

Package	Purpose
torch	DNN training and inference
torchvision	CIFAR-10 dataset + transforms
flask	REST APIs for service and vault
requests	HTTP communication between nodes
numpy	Dirichlet partitioning