Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion environment-setup/configuration.mdx
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
---
title: "Configuration"
description: "Customize your tracebloc workspace — environment variables, cluster management, GPU support, and manual Helm deployment."

Check warning on line 3 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L3

Did you really mean 'tracebloc'?
---

The installer uses sensible defaults; this page covers what you can change.
Expand DownExpand Up@@ -43,7 +43,7 @@

### View logs

The jobs manager is the main tracebloc process. Check its logs when debugging connectivity or job execution issues:

Check warning on line 46 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L46

Did you really mean 'tracebloc'?

```bash
kubectl logs -n <workspace> -l app=manager
Expand DownExpand Up@@ -71,7 +71,7 @@

Fully automatic. The installer:

1. Detects NVIDIA GPUs via `nvidia-smi` or `lspci`

Check warning on line 74 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L74

Did you really mean 'GPUs'?
2. Installs drivers if missing (Ubuntu, RHEL/CentOS, Arch)
3. Installs the NVIDIA Container Toolkit and configures Docker
4. Deploys the NVIDIA k8s device plugin into the cluster
Expand All@@ -81,11 +81,11 @@

### AMD (Linux)

Auto-detected. ROCm is installed automatically on Ubuntu and RHEL/CentOS. A logout/login may be needed for full GPU access.

Check warning on line 84 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L84

Did you really mean 'ROCm'?

### macOS

CPU only. Docker Desktop on macOS does not support GPU passthrough. For GPU workloads, deploy on a Linux machine with NVIDIA GPUs or use [AWS (EKS)](/environment-setup/eks-client-deployment-guide).

Check warning on line 88 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L88

Did you really mean 'passthrough'?

Check warning on line 88 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L88

Did you really mean 'GPUs'?

### Windows

Expand DownExpand Up@@ -158,7 +158,7 @@
```

<Note>
**The database must stay on local disk.** MySQL/InnoDB is unsafe on NFS/CIFS, so the database and logs always use the local `/tracebloc` tree. To place large **datasets** on a network mount, set the installer's `HOST_DATASET_DIR` — it relocates only the dataset volume (the chart's `hostPath.datasetPath` → `/tracebloc-data`) and runs ingestion as the mount's owner uid, so writes succeed under NFS `root_squash`.

Check warning on line 161 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L161

Did you really mean 'uid'?
</Note>

Platform snippets (drop into your values file):
Expand DownExpand Up@@ -247,7 +247,7 @@

#### Docker Registry

The chart pulls the client image from a container registry — credentials are required in production. Use a token, not a plaintext password.

Check warning on line 250 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L250

Did you really mean 'plaintext'?

```yaml
dockerRegistry:
Expand DownExpand Up@@ -284,11 +284,11 @@
timeout: "10m"
```

The CronJob's ServiceAccount is bound to the built-in `cluster-admin` ClusterRole because the chart templates cluster-scoped resources (PriorityClass, StorageClass, ClusterRoleBinding, optionally Namespace). Disable if you need a manual approval gate on upgrades.

Check warning on line 287 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L287

Did you really mean 'Namespace'?

#### NetworkPolicy hardening for training pods

Training pods run untrusted ML code. The chart can apply a NetworkPolicy that denies ingress and restricts egress to DNS + external HTTPS only — blocking pod-to-pod, MySQL, and Kubernetes API access from the training pod.
Training pods run untrusted ML code. The chart can apply a NetworkPolicy that denies all ingress and restricts egress — arbitrary pod-to-pod traffic and the Kubernetes API are blocked, while the in-cluster MySQL that serves the training data and the proxy that reports results stay reachable. Direct outbound HTTPS is allowed by default; a stricter lockdown that instead routes it through an in-cluster egress gateway can be enabled per cluster.

Check warning on line 291 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L291

Did you really mean 'untrusted'?

Check warning on line 291 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L291

Did you really mean 'lockdown'?

```yaml
networkPolicy:
Expand All@@ -314,12 +314,12 @@
Leave `enabled: false` on clusters without an enforcing CNI — silently having no protection is worse than explicitly disabling it.

<Warning>
The chart's training-pod egress lockdown only blocks traffic if your CNI enforces NetworkPolicy. Verify your CNI before relying on it.

Check warning on line 317 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L317

Did you really mean 'lockdown'?
</Warning>

#### Resource Monitor and node-agents namespace

Check warning on line 320 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L320

Did you really mean 'namespace'?

The `tracebloc-resource-monitor` DaemonSet collects node-level CPU/memory metrics. It mounts `hostPath` volumes (`/proc`, `/sys`) which Pod Security Admission's `restricted` profile bans — so the chart isolates it in a dedicated **privileged** namespace (default `tracebloc-node-agents`).

Check warning on line 322 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L322

Did you really mean 'namespace'?

```yaml
resourceMonitor: true # set false on clusters where metrics-server cannot be installed
Expand All@@ -329,7 +329,7 @@
name: tracebloc-node-agents
```

When `create: false`, create the namespace yourself with the required PSA labels:

Check warning on line 332 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L332

Did you really mean 'namespace'?

```bash
kubectl create namespace tracebloc-node-agents
Expand All@@ -343,7 +343,7 @@

#### Pod Security Admission labels

Training Jobs run untrusted user-supplied ML code. The chart can create the release namespace with Pod Security Admission `warn`/`audit`/`enforce` labels at the `restricted` profile for defense-in-depth:

Check warning on line 346 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L346

Did you really mean 'untrusted'?

Check warning on line 346 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L346

Did you really mean 'namespace'?

```yaml
namespace:
Expand All@@ -354,7 +354,7 @@
enforce: restricted # set "" for bare-metal hostPath installs
```

When `create: false` (default) and you want PSA labels on an existing namespace:

Check warning on line 357 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L357

Did you really mean 'namespace'?

```bash
kubectl label namespace <workspace> \
Expand All@@ -379,7 +379,7 @@

#### PriorityClass and PodDisruptionBudgets

The chart pins the MySQL pod with a `tracebloc-data-plane` PriorityClass (value `1000000`) so it survives node-level OOM and scheduling pressure, and applies PDBs to MySQL and the jobs manager. Override only if you run a multi-replica MySQL externally:

Check warning on line 382 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L382

Did you really mean 'PDBs'?

```yaml
priorityClass:
Expand All@@ -394,7 +394,7 @@

### Deploy

Install the chart into a new namespace:

Check warning on line 397 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L397

Did you really mean 'namespace'?

```bash
helm upgrade --install <workspace> tracebloc/client \
Expand DownExpand Up@@ -425,7 +425,7 @@
helm uninstall <workspace> -n <workspace>
```

PVCs and the PriorityClass are annotated `helm.sh/resource-policy: keep` so your data and shared cluster resources survive uninstall. To remove them too:

Check warning on line 428 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L428

Did you really mean 'PVCs'?

```bash
kubectl delete pvc --all -n <workspace>
Expand All@@ -445,11 +445,11 @@

## Security

Tracebloc is designed so your data never has to leave your network. Here's how:

Check warning on line 448 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L448

Did you really mean 'Tracebloc'?

- **Data stays local.** Training data never leaves your infrastructure. Only metadata and metrics are shared with the platform.
- **Encrypted.** All communication between client and platform is TLS-encrypted.
- **Isolated.** Training runs in containers with restricted system access. Kubernetes namespaces separate workloads from each other.

Check warning on line 452 in environment-setup/configuration.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/configuration.mdx#L452

Did you really mean 'namespaces'?
- **Scanned.** Submitted models are analyzed for vulnerabilities before execution on your infrastructure.
- **Minimal footprint.** The installer only modifies `~/.tracebloc/` and Docker. No system-wide changes.

2 changes: 1 addition & 1 deletion environment-setup/eks-client-deployment-guide.mdx
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
---
title: "Amazon EKS"
description: "Deploy a tracebloc workspace on Amazon EKS using the AWS CLI — networking, GPU support, storage, and security for a production cluster."

Check warning on line 3 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L3

Did you really mean 'tracebloc'?
---

## Overview

<Note>
**Use EKS for production** — multi-node, autoscaling, or shared GPU clusters on AWS. For a single machine (a laptop or one server), the [local installer](/environment-setup/setup-guide) is simpler and faster.

Check warning on line 9 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L9

Did you really mean 'autoscaling'?
</Note>

Running machine learning workloads in the cloud often requires a reliable, secure, and scalable infrastructure—yet setting it up can be complex. This guide walks you through building a complete Amazon EKS (Elastic Kubernetes Service) environment from scratch using the AWS CLI. By following these steps, you'll create a production-ready foundation with networking, GPU-optional compute, storage, and security fully aligned with AWS and Kubernetes best practices.

Once the infrastructure is in place, you'll deploy and configure the tracebloc client to securely train and benchmark AI models. This setup ensures that your proprietary data stays within your environment, while still allowing external AI models to be tested and fine-tuned in a controlled, isolated way. The result: a scalable, secure platform for high-performance ML workloads that accelerates collaboration with external experts while maintaining full control over your data and IP.

Check warning on line 14 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L14

Did you really mean 'tracebloc'?

The entire setup takes ~1–2 hours.

Expand All@@ -25,7 +25,7 @@

#### 1. Create or use an AWS Account
- If you don't already have one, [create an AWS account](https://signin.aws.amazon.com/signup?request_type=register).
- Your account must have permissions to create and manage EKS resources, VPCs, EC2 instances, and IAM roles.

Check warning on line 28 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L28

Did you really mean 'VPCs'?

#### 2. Install and configure AWS CLI
- [Install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) on your local machine.
Expand DownExpand Up@@ -53,10 +53,10 @@
- EC2 instances and security groups
- IAM roles and policies

### Required Tooling & Tracebloc Account

Check warning on line 56 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L56

Did you really mean 'Tracebloc'?
- **Helm 3.x**: Install Helm on your local machine. [Installation Guide](https://helm.sh/docs/intro/install/)
- **kubectl**: Install kubectl to interact with your EKS cluster. [Installation Guide](https://kubernetes.io/docs/tasks/tools/)
- **Tracebloc Account**: You will need your Client ID and Client Password ([from the tracebloc client view](https://ai.tracebloc.io/clients)).

Check warning on line 59 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L59

Did you really mean 'Tracebloc'?
- **Docker Registry Credentials**: Docker Hub username, password/token, and email for pulling container images.

### Recommended for Monitoring: Use k9s
Expand All@@ -66,15 +66,15 @@

### Components

The EKS infrastructure setup consists of two main parts: Core Infrastructure and Client Deployment. Together, they provide a secure, scalable environment for running ML workloads with the tracebloc client.

Check warning on line 69 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L69

Did you really mean 'tracebloc'?

#### Core Infrastructure

**VPC with isolated networking**
Creates a dedicated, secure network environment with subnets across multiple Availability Zones for high availability.

Check warning on line 74 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L74

Did you really mean 'subnets'?

**EKS Cluster**
A managed Kubernetes control plane with auto-scaling nodegroups to run your ML workloads.

Check warning on line 77 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L77

Did you really mean 'nodegroups'?

**IAM Roles and Policies**
Granular roles for the cluster and worker nodes following least-privilege principles.
Expand All@@ -87,7 +87,7 @@

#### Client Deployment

**tracebloc Client Application**

Check warning on line 90 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L90

Did you really mean 'tracebloc'?
Deployed into the EKS cluster using Helm, configured with your account credentials and storage.

**Monitoring and Verification**
Expand All@@ -98,7 +98,7 @@
### Process Flow

1. **Infrastructure Setup** - Create VPC, networking, EKS cluster, and storage
2. **Client Deployment** - Configure and deploy the tracebloc application

Check warning on line 101 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L101

Did you really mean 'tracebloc'?
3. **Verification** - Validate everything is running correctly


Expand All@@ -113,26 +113,26 @@
- **Model Encryption**: Model weights are encrypted and never leave your environment.
- **Secure Communication**: All platform communication is protected with TLS.

### tracebloc Backend Security

Check warning on line 116 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L116

Did you really mean 'tracebloc'?

- **Code Analysis**: Submitted external models are scanned with the Bandit library to detect vulnerabilities or malicious code.
- **Input Validation**: Training scripts and model code undergo strict validation before execution.
- **Sandboxed Execution**: Training workloads run in isolated environments with restricted system access.

Check warning on line 120 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L120

Did you really mean 'Sandboxed'?
- **Namespace Isolation**: Kubernetes namespaces ensure logical separation from other applications.

Check warning on line 121 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L121

Did you really mean 'Namespace'?

Check warning on line 121 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L121

Did you really mean 'namespaces'?

### Identity & Access Management

**AWS IAM Roles:**
- **Cluster Role**: Minimal permissions (`AmazonEKSClusterPolicy`) for cluster management.
- **Nodegroup Role**: Least-privilege permissions for worker nodes, networking, container registry, and storage access.

Check warning on line 127 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L127

Did you really mean 'Nodegroup'?

**Kubernetes RBAC:**
- Cluster roles provide limited permissions for jobs, pods, and deployments.
- Service accounts are bound to roles within their namespace, preventing cross-namespace access.

Check warning on line 131 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L131

Did you really mean 'namespace'?

### Network Security

- **VPC Isolation**: Dedicated VPC with private subnets and an Internet Gateway for controlled communication.

Check warning on line 135 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L135

Did you really mean 'subnets'?
- **Security Groups**: Restrict access to EFS mount targets and cluster traffic.
- **Outbound Access**: Limited to required destinations such as `*.amazonaws.com`, `*.docker.io`, and `*.tracebloc.io`.

Expand All@@ -143,13 +143,13 @@
Quick Setup runs an automated script that builds the whole cluster in one go. Want step-by-step control (or to customize networking)? Use [Detailed Setup](#detailed-setup) instead.

### Purpose
Spin up a production-ready EKS baseline (VPC, subnets, internet gateway, EKS cluster, managed nodegroup, EFS + CSI driver) in one go. Includes basic validation, colored logging, and a cleanup mode.

Check warning on line 146 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L146

Did you really mean 'subnets'?

Check warning on line 146 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L146

Did you really mean 'nodegroup'?

### What this script does
- Creates networking (VPC + 3 public subnets + route to IGW)

Check warning on line 149 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L149

Did you really mean 'subnets'?
- Provisions an EKS cluster and a system nodegroup (t3.medium, 2–5 nodes)

Check warning on line 150 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L150

Did you really mean 'nodegroup'?
- Creates EFS and mount targets in all AZs, sets up the EFS CSI driver + storage class

Check warning on line 151 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L151

Did you really mean 'AZs'?
- Updates your kubeconfig for kubectl access

Check warning on line 152 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L152

Did you really mean 'kubeconfig'?
- Prints a summary and next steps

### Prerequisites
Expand All@@ -165,34 +165,34 @@
4. Execute: `./setup_eks.sh`
5. When finished, follow the printed "Next steps" to create your Docker registry secret and deploy workloads

### Cleanup (teardown)

Check warning on line 168 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L168

Did you really mean 'teardown'?
Run `./setup_eks.sh cleanup` to remove cluster, nodegroup, EFS, subnets, gateway, roles, and VPC (irreversible).

Check warning on line 169 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L169

Did you really mean 'nodegroup'?

Check warning on line 169 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L169

Did you really mean 'subnets'?

### Tips:
- **Costs**: This creates billable resources (EC2, EKS, EFS, data transfer). Remove when not needed.
- **Network model**: Subnets are configured to auto-assign public IPs for simplicity. Adjust to private subnets + NAT as needed.

Check warning on line 173 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L173

Did you really mean 'Subnets'?

Check warning on line 173 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L173

Did you really mean 'IPs'?

Check warning on line 173 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L173

Did you really mean 'subnets'?
- **Kubernetes version**: The script requests `--kubernetes-version 1.32`; update if your region/account supports a different current version.
- **Security hardening**: This is a production baseline; harden further for your environment (security groups, private subnets, IRSA, Pod Security/OPA).

Check warning on line 175 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L175

Did you really mean 'subnets'?

If you prefer more control over your setup and want to customize the environment to your needs, follow the step-by-step guide below.

## Detailed Setup

This section walks through a step-by-step build with AWS CLI and kubectl. It mirrors the Quick Setup but lets you choose your own resource settings (# of CPUs, Memory, VPC CIDRs, instance types, namespace, Helm release name, StorageClass, etc.). Expect about 1–2 hours end-to-end.

Check warning on line 181 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L181

Did you really mean 'CPUs'?

Check warning on line 181 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L181

Did you really mean 'CIDRs'?

Check warning on line 181 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L181

Did you really mean 'namespace'?

### What you'll do (Steps 1–6):
1. **VPC & Network Configuration** — Create an isolated Virtual Private Cloud (VPC) with subnets distributed across three availability zones for high availability, an Internet Gateway for outbound connectivity, and routing tables with proper associations. This provides network isolation and fault tolerance for your cluster infrastructure.

Check warning on line 184 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L184

Did you really mean 'subnets'?
2. **EKS Cluster Setup** — Create the cluster service role and provision the EKS control plane which manages the Kubernetes API server.
3. **EKS Nodegroup Setup** — Create a node service role and provision two managed nodegroups: A system nodegroup for Kubernetes system components and a training nodegroup for ML workloads. Each nodegroup can be customized with different instance types, scaling parameters, and capacity types depending on the work loads and data types.

Check warning on line 186 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L186

Did you really mean 'Nodegroup'?

Check warning on line 186 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L186

Did you really mean 'nodegroups'?

Check warning on line 186 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L186

Did you really mean 'nodegroup'?

Check warning on line 186 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L186

Did you really mean 'nodegroup'?

Check warning on line 186 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L186

Did you really mean 'nodegroup'?
4. **Storage** — Create an Amazon EFS file system for shared persistent storage, configure a security group that allows NFS traffic from the cluster nodes, and create mount targets in each availability zone. This provides scalable, shared storage for training data as well as weights and logs.
5. **Client Configuration** — Install the Amazon EFS (Elastic File System) CSI driver (Container Storage Interface) in your EKS (Elastic Kubernetes Service) cluster. This driver is what lets Kubernetes automatically create and mount EFS storage volumes.
6. **Client Deployment** — Add the tracebloc Helm repository, configure your deployment values (authentication credentials, registry access, storage settings, resource limits), install the chart into your chosen namespace. Deploy and verify that all pods are running and persistent volume claims are properly bound.

Check warning on line 189 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L189

Did you really mean 'tracebloc'?

Check warning on line 189 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L189

Did you really mean 'namespace'?

**Helm Usage**: Helm is used to install and manage Kubernetes applications. In steps 5 and 6 you will deploy the tracebloc client via the unified `tracebloc/client` chart with EKS-specific values.

Check warning on line 191 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L191

Did you really mean 'tracebloc'?

## 1. VPC and Network Configuration

Your AWS EKS cluster must run in a secure and isolated network with subnets across availability zones, DNS resolution, and internet access. This is provided by an AWS VPC (Virtual Private Cloud), which is a logically isolated section of AWS where you define your own IP address range (CIDR block), subnets, routing, and gateways.

Check warning on line 195 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L195

Did you really mean 'subnets'?

Check warning on line 195 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L195

Did you really mean 'subnets'?
This section sets up the foundation.

### VPC Creation
Expand All@@ -207,20 +207,20 @@

Creates an isolated network environment where your EKS cluster runs securely. Uses CIDR (Classless Inter-Domain Routing) block `10.0.0.0/16` to define the IP address range for all resources in the VPC and tags it with the specified owner name. Replace `<OWNER_NAME>` with your preferred identifier. Keep the output and the `VPC_ID`for the next steps.

### Enable DNS Hostnames

Check warning on line 210 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L210

Did you really mean 'Hostnames'?

DNS (Domain Name System) support and hostnames let Kubernetes pods and services find each other by name, which is essential for service discovery in Kubernetes, which relies on DNS resolution.

Check warning on line 212 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L212

Did you really mean 'hostnames'?

```bash
aws ec2 modify-vpc-attribute --vpc-id <VPC_ID> --enable-dns-hostnames
```

Enables DNS resolution so nodes and services can resolve each other by name instead of IP addresses. Find the `VPC_ID`from the previous step or list VPCs using `aws ec2 describe-vpcs`.

Check warning on line 218 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L218

Did you really mean 'VPCs'?


### Create Subnets Across Availability Zones

Check warning on line 221 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L221

Did you really mean 'Subnets'?

Subnets (smaller ranges of IP addresses inside the VPC) are spread across multiple AZs (Availability Zones) to ensure high availability and fault tolerance. EKS requires subnets in at least 2 availability zones for high availability - using 3 zones provides better fault tolerance and load distribution.

Check warning on line 223 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L223

Did you really mean 'Subnets'?

Check warning on line 223 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L223

Did you really mean 'AZs'?

Check warning on line 223 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L223

Did you really mean 'subnets'?

```bash
aws ec2 create-subnet \
Expand All@@ -242,18 +242,18 @@
--tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=<OWNER_NAME>}]"
```

Creates three subnets with separate CIDR blocks (`10.0.1.0/24`, `10.0.2.0/24`, `10.0.3.0/24`) distributed across different availability zones. Use or create tags if needed. Register the generated `SUBNET_IDs` for the following steps.

Check warning on line 245 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L245

Did you really mean 'subnets'?

### Enable Public IPs on Subnets

Check warning on line 247 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L247

Did you really mean 'IPs'?

Check warning on line 247 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L247

Did you really mean 'Subnets'?

EKS worker nodes need public IPs to communicate with the EKS control plane, download container images from Dockerhub, and allow external traffic to reach applications.

Check warning on line 249 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L249

Did you really mean 'IPs'?

Check warning on line 249 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L249

Did you really mean 'Dockerhub'?

```bash
aws ec2 modify-subnet-attribute --subnet-id <SUBNET_ID_1A> --map-public-ip-on-launch
aws ec2 modify-subnet-attribute --subnet-id <SUBNET_ID_1B> --map-public-ip-on-launch
aws ec2 modify-subnet-attribute --subnet-id <SUBNET_ID_1C> --map-public-ip-on-launch
```
Configures each subnet to automatically assign public IP addresses to new EC2 instances.

Check warning on line 256 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L256

Did you really mean 'subnet'?

### Create and Attach an Internet Gateway

Expand DownExpand Up@@ -300,7 +300,7 @@
## 2. EKS Cluster Setup

Amazon EKS provides the managed Kubernetes control plane. It needs permissions through AWS IAM (Identity and Access Management) to create and manage resources such as load balancers, security groups, and network interfaces.
The cluster itself is the logical container for the API server, linking your VPC networking with the control plane and IAM role. To interact with it you must update your local kubeconfig so kubectl can send commands. Setting this up gives you a working API server that can manage workloads once worker nodes are added in the next step.

Check warning on line 303 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L303

Did you really mean 'kubeconfig'?

### Create EKS Cluster Role

Expand DownExpand Up@@ -339,7 +339,7 @@

### Create EKS Cluster

Create the EKS cluster to provision the managed Kubernetes control plane and connect it to your VPC, subnets, and IAM role.

Check warning on line 342 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L342

Did you really mean 'subnets'?

```bash
aws eks create-cluster \
Expand All@@ -350,7 +350,7 @@
--kubernetes-version 1.32 \
--tags owner=<OWNER_NAME>
```
Creates the EKS cluster using the specified role, VPC subnets, and Kubernetes version. Set an appropriate `CLUSTER_NAME`, **expect this to take around 10 minutes**. If you do not have your accound ID ready, run `aws sts get-caller-identity`.

Check warning on line 353 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L353

Did you really mean 'subnets'?

Check warning on line 353 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L353

Did you really mean 'accound'?

### Configure kubectl

Expand All@@ -359,7 +359,7 @@
```bash
aws eks update-kubeconfig --region eu-central-1 --name <CLUSTER_NAME>
```
Updates your local kubeconfig for kubectl.

Check warning on line 362 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L362

Did you really mean 'kubeconfig'?

### Manage Multiple Clusters (Optional)

Expand All@@ -375,22 +375,22 @@
```
This ensures subsequent kubectl commands apply to the intended cluster.

### Create Application Namespace

Check warning on line 378 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L378

Did you really mean 'Namespace'?

Namespaces let you isolate your workloads from system components and other applications running in the cluster. Define an appropriate namespace for your setup

Check warning on line 380 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L380

Did you really mean 'Namespaces'?

Check warning on line 380 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L380

Did you really mean 'namespace'?

```bash
kubectl create namespace <NAMESPACE>
```
Adds a dedicated namespace for your workloads, keeping them isolated from system components.

Check warning on line 385 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L385

Did you really mean 'namespace'?



## 3. EKS Nodegroup Setup

Check warning on line 389 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L389

Did you really mean 'Nodegroup'?

Nodegroups are sets of EC2 instances that act as worker nodes, running your system and training workloads. Each nodegroup needs an IAM role so the nodes can join the cluster and interact with AWS services such as pulling images from registries, attaching EFS storage, and sending logs. It is best practice to create at least two groups: a small system nodegroup to host cluster services and a training nodegroup sized for your ML workloads. The training nodegroup can be CPU-based for general jobs or GPU-based for deep learning, ensuring that heavy jobs do not interfere with core Kubernetes functions.

Check warning on line 391 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L391

Did you really mean 'Nodegroups'?

Check warning on line 391 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L391

Did you really mean 'nodegroup'?

Check warning on line 391 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L391

Did you really mean 'nodegroup'?

Check warning on line 391 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L391

Did you really mean 'nodegroup'?

Check warning on line 391 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L391

Did you really mean 'nodegroup'?

### Create Nodegroup Role

Check warning on line 393 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L393

Did you really mean 'Nodegroup'?

Worker nodes need an IAM role to join the cluster and access AWS services.

Expand DownExpand Up@@ -432,11 +432,11 @@
- **AmazonEC2ContainerRegistryReadOnly:** allows pulling images from ECR
- **AmazonEKS_CNI_Policy:** required for pod networking via the Container Network Interface (CNI) plugin

### Create Managed Nodegroups

Check warning on line 435 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L435

Did you really mean 'Nodegroups'?

Now let`s create two nodegroups: A system nodegroup that runs critical Kubernetes components (CoreDNS, kube-proxy, Container Network Interface (CNI), Storage Interface drivers (CSI), metrics-server, etc.) and a training nodegroup that runs your machine learning workloads. This separation ensures cluster stability even under heavy training load.

Check warning on line 437 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L437

Did you really mean 'nodegroups'?

Check warning on line 437 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L437

Did you really mean 'nodegroup'?

Check warning on line 437 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L437

Did you really mean 'nodegroup'?

#### System Nodegroup

Check warning on line 439 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L439

Did you really mean 'Nodegroup'?

It is recommended to use [AWS EC2 instance type t3.medium](https://aws.amazon.com/ec2/instance-types/t3/) for system pods as they are relatively cheap and sufficient. Spread nodes across three Availability Zones for resilience, set ON_DEMAND for reliability and label as "system":

Expand All@@ -455,11 +455,11 @@
--tags owner=<OWNER_NAME>
```

Creates a nodegroup with `t3.medium` instances (2 vCPUs, 4 GiB memory) spread across three AZs. The group scales between 2 and 5 nodes (EC2s), each node is labeled for system workloads. Set an appropriate `SYSTEM_NODEGROUP_NAME`. Always make sure to use `AL2_x86_64` for system nodes so that the data ingestor runs on this node, too.

Check warning on line 458 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L458

Did you really mean 'nodegroup'?

Check warning on line 458 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L458

Did you really mean 'vCPUs'?

Check warning on line 458 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L458

Did you really mean 'AZs'?

Check warning on line 458 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L458

Did you really mean 'ingestor'?

#### Training Nodegroup

Check warning on line 460 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L460

Did you really mean 'Nodegroup'?

This group runs your ML training workloads — size it for your dataset, model type, number of parallel workloads, and whether you need GPUs.

Check warning on line 462 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L462

Did you really mean 'GPUs'?

Refer to the [EC2 instance types list](https://aws.amazon.com/ec2/instance-types) and [EKS managed nodegroups docs](https://docs.aws.amazon.com/eks/latest/userguide/create-managed-node-group.html) for guidance.

Expand All@@ -483,17 +483,17 @@

| Compute Type | INSTANCE_TYPE | AMI_TYPE | LABEL |
|--------------|---------------|----------|-------|
| CPU | t3.xlarge | AL2_x86_64 | cpu |

Check warning on line 486 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L486

Did you really mean 'cpu'?
| GPU | g5g.xlarge | AL2023_x86_64_NVIDIA | gpu |

Check warning on line 487 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L487

Did you really mean 'gpu'?

The first example provisions a training nodegroup with t3.xlarge instances (4 vCPUs, 16 GiB memory). It starts with 2 nodes, can scale down to 0 when idle, and grows to 5 under load.

Check warning on line 489 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L489

Did you really mean 'nodegroup'?

Check warning on line 489 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L489

Did you really mean 'vCPUs'?

You can limit the compute per participant or team: Refer to the [creating a use case](/create-use-case/define) section for details.


#### Optional for GPU Training Nodegroup

Check warning on line 494 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L494

Did you really mean 'Nodegroup'?

Install the NVIDIA device plugin so Kubernetes can automatically detect and schedule GPUs on any new node:

Check warning on line 496 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L496

Did you really mean 'GPUs'?

```bash
kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.17.1/deployments/static/nvidia-device-plugin.yml
Expand All@@ -506,7 +506,7 @@
- model checkpoints and weights during training
- client logs

Tracebloc leverages Amazon EFS (Elastic File System) to provide a scalable, managed file system that integrates directly with Kubernetes via the EFS CSI (Container Storage Interface) driver. It provides shared, persistent storage that all nodes and pods in the cluster can access at the same time.

Check warning on line 509 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L509

Did you really mean 'Tracebloc'?


### Create EFS File System
Expand DownExpand Up@@ -557,7 +557,7 @@
--security-groups <SECURITY_GROUP_ID>
```

Creates EFS mount targets in each subnet so nodes across all Availability Zones can attach to the same shared file system.

Check warning on line 560 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L560

Did you really mean 'subnet'?

### Attach EFS CSI Driver Policy to Node Role

Expand All@@ -568,7 +568,7 @@
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
--role-name <NODEGROUP_ROLE_NAME>
```
Attaches the AWS-managed AmazonEFSCSIDriverPolicy to your nodegroup role. This lets the EFS CSI driver running on your nodes provision and mount EFS storage for pods, so workloads can share datasets, logs, and model checkpoints across the cluster.

Check warning on line 571 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L571

Did you really mean 'nodegroup'?

### Add EFS CSI Driver Helm Repository

Expand All@@ -578,7 +578,7 @@
helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver
```

This command registers the AWS EFS CSI driver Helm repository with your local Helm setup. Once added, you can install or update the EFS CSI driver in your cluster using Helm commands. The driver runs as pods in the kube-system namespace and allows Kubernetes to dynamically provision and mount EFS volumes for your workloads.

Check warning on line 581 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L581

Did you really mean 'namespace'?

### Configure EFS CSI Driver Placement

Expand All@@ -604,11 +604,11 @@
values:
- system
```
This node affinity ensures the EFS CSI controller pods run on the system nodegroup (nodes labeled node.kubernetes.io/scope=system) instead of training nodes.

Check warning on line 607 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L607

Did you really mean 'nodegroup'?

### Install EFS CSI Driver with Helm

Now, the EFS CSI driver needs to be installed into the cluster (namespace kube-system by convention) using the configuration file.

Check warning on line 611 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L611

Did you really mean 'namespace'?

```bash
helm install efs-csi-driver aws-efs-csi-driver/aws-efs-csi-driver \
Expand All@@ -618,7 +618,7 @@

### Install Metrics Server

The Metrics Server is the cluster-wide aggregator for resource usage data. It collects CPU and memory metrics for autoscaling and live resource monitoring. Install it to monitor resources usage:

Check warning on line 621 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L621

Did you really mean 'autoscaling'?

```bash
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability-1.21+.yaml
Expand All@@ -629,12 +629,12 @@

## 5. Client Configuration

You can now prepare the Helm chart that deploys the tracebloc client itself: The client is the component that runs your model evaluation and training jobs inside the cluster. To install it, you use a Helm chart, which bundles all required Kubernetes manifests into a single, configurable package.

Check warning on line 632 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L632

Did you really mean 'tracebloc'?
A values.yaml file controls this deployment, where you specify credentials (to authenticate with tracebloc), registry access (to pull container images), and storage settings (to mount your EFS volumes). This configuration ensures the client can securely connect, schedule workloads, and store results.

Check warning on line 633 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L633

Did you really mean 'tracebloc'?

### Add Helm Repository

The tracebloc client is delivered as a single unified Helm chart (`tracebloc/client`) that supports AKS, EKS, bare-metal, and OpenShift. Source: [tracebloc/client](https://github.com/tracebloc/client).

Check warning on line 637 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L637

Did you really mean 'tracebloc'?

```bash
helm repo add tracebloc https://tracebloc.github.io/client
Expand All@@ -651,7 +651,7 @@

Open and update the following sections in `values.yaml`:

#### Tracebloc Authentication

Check warning on line 654 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L654

Did you really mean 'Tracebloc'?

Provide your Client ID and password from the [tracebloc client view](https://ai.tracebloc.io/clients):

Expand All@@ -662,7 +662,7 @@

#### Docker Registry Configuration

The tracebloc client images live on Docker Hub. Provide credentials so Kubernetes can pull them — the chart auto-creates a secret named `{{ .Release.Name }}-regcred`:

Check warning on line 665 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L665

Did you really mean 'tracebloc'?

```yaml
dockerRegistry:
Expand DownExpand Up@@ -711,7 +711,7 @@

#### Set Resource Limits for Training Jobs

Each training Job inherits these resource requests/limits. Make sure they fit within your EC2 instance capacity. **IMPORTANT:** For GPU jobs, `requests` and `limits` must be equal — Kubernetes rejects configs where they differ.

Check warning on line 714 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L714

Did you really mean 'configs'?

```yaml
env:
Expand All@@ -721,7 +721,7 @@
GPU_LIMITS: "" # for GPU support set "nvidia.com/gpu=1"
```

**Tip:** Estimating VRAM requirements for LLMs can be tricky. Use the [VRAM Calculator](https://apxml.com/tools/vram-calculator) to approximate memory needs for different model sizes and batch configurations.

Check warning on line 724 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L724

Did you really mean 'LLMs'?

**Node, Pod, Job relationship**

Expand All@@ -743,7 +743,7 @@

#### NetworkPolicy for Training Pods

The chart can apply a NetworkPolicy that denies ingress and restricts egress to DNS + external HTTPS only — blocking pod-to-pod, MySQL, and Kubernetes API access from training pods.
The chart can apply a NetworkPolicy that denies all ingress and restricts egress — arbitrary pod-to-pod traffic and the Kubernetes API are blocked, while the in-cluster MySQL that serves the training data and the proxy that reports results stay reachable. Direct outbound HTTPS is allowed by default; a stricter lockdown that instead routes it through an in-cluster egress gateway can be enabled per cluster.

Check warning on line 746 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L746

Did you really mean 'lockdown'?

```yaml
networkPolicy:
Expand All@@ -769,11 +769,11 @@

## 6. Client Deployment

With the configuration ready, deploy the tracebloc client into your EKS cluster using Helm.

Check warning on line 772 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L772

Did you really mean 'tracebloc'?

### Deploy the client with Helm

Install the chart into your namespace using your customized values file:

Check warning on line 776 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L776

Did you really mean 'namespace'?

```bash
helm install <RELEASE_NAME> tracebloc/client \
Expand All@@ -785,12 +785,12 @@
This creates:

- One MySQL pod: `mysql-...`
- One tracebloc client pod: `<release>-jobs-manager-...`

Check warning on line 788 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L788

Did you really mean 'tracebloc'?
- A `tracebloc-resource-monitor` DaemonSet in the `tracebloc-node-agents` namespace

Check warning on line 789 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L789

Did you really mean 'namespace'?
- A `<release>-auto-upgrade` CronJob for daily chart upgrades (see [Configuration → Auto-upgrade](/environment-setup/configuration#auto-upgrade-on-by-default))
- Supporting resources: Service, ConfigMap, Secrets, PVCs, PriorityClass, PDBs

Check warning on line 791 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L791

Did you really mean 'PVCs'?

Check warning on line 791 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L791

Did you really mean 'PDBs'?

Expect a few minutes for pods to pull images, bind PVCs, and reach `Running`.

Check warning on line 793 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L793

Did you really mean 'PVCs'?


## Verification and Maintenance
Expand All@@ -798,7 +798,7 @@
After deployment, confirm the client is running correctly and learn how to maintain it.

### Verify Deployment
Check that all pods are running in your namespace:

Check warning on line 801 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L801

Did you really mean 'namespace'?

#### Check pod status:
```bash
Expand All@@ -816,7 +816,7 @@
```bash
kubectl get pvc -n <NAMESPACE>
```
Verifies that PVCs are bound and storage is available.

Check warning on line 819 in environment-setup/eks-client-deployment-guide.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/eks-client-deployment-guide.mdx#L819

Did you really mean 'PVCs'?

### Maintenance

Expand Down
2 changes: 1 addition & 1 deletion environment-setup/overview.mdx
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
---
title: "Overview"
description: "How tracebloc runs on your infrastructure — and why your data never leaves it."

Check warning on line 3 in environment-setup/overview.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/overview.mdx#L3

Did you really mean 'tracebloc'?
---

**Models come to your data — your data never leaves your infrastructure.** You run a tracebloc *workspace* on your own hardware. Contributors submit models that are scanned and run in isolation against your data, on your machines. Only results leave — and trained model weights only if you choose to share them.

Check warning on line 6 in environment-setup/overview.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/overview.mdx#L6

Did you really mean 'tracebloc'?

## The trust boundary

<Frame>
![How tracebloc keeps your data on your infrastructure — models come to the data, only results leave](/images/traceblocfederatedlearning.png)

Check warning on line 11 in environment-setup/overview.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/overview.mdx#L11

Did you really mean 'tracebloc'?
</Frame>

Raw data stays inside your infrastructure. Your workspace opens an **outbound-only** connection — nothing reaches in to pull your data out.
Expand DownExpand Up@@ -45,7 +45,7 @@
| Training & evaluation results | Yes — the metrics models are judged on |
| Trained model weights | **Only if you allow it** — your choice per collaboration, set in the admin panel |

Enforced by per-job container isolation, a NetworkPolicy that blocks data egress from training pods, a vulnerability scan before any model runs, and TLS on all traffic.
Enforced by per-job container isolation, a NetworkPolicy that restricts training-pod egress, a vulnerability scan before any model runs, and TLS on all traffic.

<Note>
**The mental model:** `1 machine = 1 workspace = n datasets`. One deployment per machine; as many datasets inside it as you like.
Expand All@@ -55,7 +55,7 @@

| Term | What it is |
|---|---|
| **Workspace** | Your private, dedicated AI environment — tracebloc's software running on your own infrastructure, where you invite contributors to train models on your data |

Check warning on line 58 in environment-setup/overview.mdx

View check run for this annotation

Mintlify/ Mintlify Validation (tracebloc) - vale-spellcheck

environment-setup/overview.mdx#L58

Did you really mean 'tracebloc's'?
| **Client ID** | The credential that connects your workspace to the platform (created on the clients page) |
| **Dataset** | Data you've staged locally for use cases |
| **Use case** | A task contributors build models for, against your datasets |
Expand Down
Loading