Skip to content

Client v2 - #16

Merged
saadqbal merged 50 commits into
developfrom
client-v2
Mar 4, 2026
Merged

Client v2#16
saadqbal merged 50 commits into
developfrom
client-v2

Conversation

@saadqbal

@saadqbalsaadqbal commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

Note

Medium Risk
Medium risk due to substantial Helm chart template/RBAC/secret-name changes across multiple deployment targets plus new GitHub Actions release automation, which could break existing installs or publishing if misconfigured.

Overview
Adds a new unified Helm chart under tracebloc/ (v2.0.0) with platform-specific CI values, migration docs, and production install documentation, and wires in GitHub Actions to lint/template/schema-test the chart on PRs and package/publish it to gh-pages and GitHub Releases on tags.

Updates the legacy per-platform charts (aks/, bm/, eks/, oc/) with standard labels, dedicated ServiceAccounts, stricter secret handling (required client password), optional node selectors, PVC helm.sh/resource-policy: keep, MySQL split into dedicated resources with probes, tighter StorageClass permissions, and more scoped RBAC rules; adds DEPRECATED.md notices and OpenShift guidance.

Introduces one-command local Kubernetes setup scripts (scripts/install.sh + modular libs, and Windows install.ps1/install-k8s.ps1) that install Docker/k3d/kubectl/Helm, detect GPUs, and optionally install/configure NVIDIA/AMD components and device plugins.

Written by Cursor Bugbot for commit c72a8a0. This will update automatically on new commits. Configure here.

- Updated .gitignore to include environment-specific secrets and Helm deployment value files.
- Added MySQL PersistentVolumeClaim and ConfigMap for configuration management.
- Introduced MySQL Deployment and Service templates for better database management.
- Enhanced jobs-manager-deployment with security context, resource requests, and environment variable management.
- Updated RBAC roles to include service account creation and improved access control.
- Added resource monitor DaemonSet configuration and associated settings.
- Improved documentation in NOTES.txt for deployment verification and component overview.
- Introduced a new Helm CI workflow for automated linting, templating, and validation across multiple platforms.
- Added deprecation notices for legacy charts (aks, bm, eks, oc) in favor of the unified tracebloc chart.
- Updated templates to utilize a centralized registry secret name for improved consistency.
- Enhanced values schema and default values for better configuration management.
- Added tests for jobs manager and MySQL deployment to ensure template integrity and functionality.
- Improved documentation in values.yaml and NOTES.txt for clarity on deployment configurations.
…m Helm chart
- Eliminated `imageRegistry` and `nodeSelector` settings from various configuration files to streamline deployment.
- Updated templates to default to `docker.io` for image references, ensuring consistency across environments.
- Added PodDisruptionBudgets for jobs manager and MySQL deployments to enhance availability during maintenance.
- Improved deployment configurations with termination grace periods for better resource management.
- Enhanced documentation in NOTES.txt for clearer deployment instructions and component overview.
- Updated `MIGRATION.md` to reflect changes in cluster role configuration, replacing `useClusterScope` with `clusterScope`.
- Revised `values.schema.json` to streamline required properties and enhance descriptions for environment variables, PVCs, and RBAC settings.
- Modified `values.yaml` to consolidate environment variable settings and clarify PVC configurations.
- Enhanced template files to utilize new helper functions for PVC names and storage sizes, ensuring consistency across deployments.
- Improved test configurations to align with updated schema and values, ensuring robust validation of deployments.
Comment threadREADME.md Outdated
Comment threadmysql.yaml
Comment threadtracebloc/templates/jobs-manager-pdb.yaml Outdated
Comment threadtracebloc/templates/init-container.yaml
Comment threadtracebloc/templates/_helpers.tpl
- Introduced `install-k8s.sh` for a one-command Kubernetes installation on macOS and Linux, utilizing k3d for lightweight clusters.
- Added various helper scripts in the `lib` directory for GPU detection, driver installation, and cluster management.
- Enhanced `.gitignore` to exclude unnecessary files while including the `scripts/lib` directory.
- Implemented logging and utility functions for better user feedback during installation and setup processes.
- Provided a summary script to display cluster status and common commands post-installation.
Comment threadtracebloc/tests/platform_test.yaml
Comment threadtracebloc/templates/shared-images-pvc.yaml
- Added creation of host data directory in `_create_new_cluster` if it doesn't exist.
- Updated `HOST_DATA_DIR` default value to `$HOME/.tracebloc` for better user experience.
- Modified summary output to include volume mount information for clarity.
- Revised `MIGRATION.md` to reflect new host path structure for data, logs, and MySQL.
- Updated `values.schema.json` to change property names from `dataPath`, `logsPath`, and `mysqlPath` to `dataDir`, `logsDir`, and `mysqlDir` for consistency.
- Enhanced template files to utilize new helper functions for host path management in PVC configurations.
Comment threadtracebloc/values.yaml
Comment threadoc/templates/resource-monitor-daemonset.yaml
- Added functions to detect Ubuntu codename and RHEL version for dynamic package retrieval.
- Implemented a method to scrape the ROCm repository for the latest .deb and .rpm files.
- Improved error handling for unsupported distributions and missing packages.
- Updated installation logic to support both Ubuntu and RHEL/CentOS systems more effectively.
- name: GPU_REQUESTS
value: {{ if hasKey .Values.env "GPU_REQUESTS" }}{{ .Values.env.GPU_REQUESTS | quote }}{{ else }}"nvidia.com/gpu=1"{{ end }}
- name: GPU_LIMITS
value: {{ if hasKey .Values.env "GPU_LIMITS" }}{{ .Values.env.GPU_LIMITS | quote }}{{ else }}"nvidia.com/gpu=1"{{ end }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPU resources default changed, breaks non-GPU clusters

High Severity

When env.GPU_REQUESTS and env.GPU_LIMITS are not provided (the default, since env is {}), the template falls back to "nvidia.com/gpu=1" for both. All four legacy charts defaulted to empty strings for these values, meaning no GPU was requested. The unified chart now causes every spawned job to request an NVIDIA GPU by default. On clusters without GPU nodes — which is the common case — spawned jobs will be stuck in Pending forever because the scheduler can't satisfy the nvidia.com/gpu resource request.

Fix in CursorFix in Web

- Updated package manager installation commands to include options for handling configuration file changes during upgrades.
- Set environment variables to ensure non-interactive installations and manage restart behavior during the setup process.
…onality
- Introduced `_merge_kubeconfig` to handle kubeconfig updates and context switching.
- Added `_wait_for_api` to ensure the Kubernetes API server is reachable before proceeding.
- Streamlined the `create_cluster` function by delegating kubeconfig merging and API readiness checks to the new helper functions.
- Introduced claimRef with name and namespace fields in logs-pvc.yaml, mysql-storage-pvc.yaml, and shared-images-pvc.yaml to enhance the association of PersistentVolumeClaims with their respective resources.
- Revised the README to include a quick installation guide for setting up a local Kubernetes cluster with GPU support using a single command.
- Introduced a new `install.sh` script that downloads necessary installation scripts and executes the Kubernetes installation process.
- Updated the `install-k8s.sh` script reference in the comments to point to the new bootstrap installer for clarity.
- Modified the `install.sh` script to allow users to specify a branch for downloading the installer, defaulting to 'main'.
- Enhanced the download message to indicate the selected branch for better user clarity.
- Updated the Docker installation process to force install the application if not already present.
- Improved waiting mechanism for Docker engine startup with a configurable maximum wait time.
- Added user guidance for accepting the Docker license agreement on first launch to prevent errors during setup.
- Introduced a new `install.ps1` script for Windows that downloads and executes the `install-k8s.ps1` script.
- Updated `install.sh` to include instructions for Windows users, directing them to use the new PowerShell installer.
- Enhanced error handling and retry logic in various scripts to improve robustness during downloads.
- Improved help messages and usage instructions across installation scripts for better user guidance.
- Standardized comment formatting and improved clarity in usage instructions.
- Enhanced error messages for better user feedback during installation.
- Updated logging and banner display for a more consistent user experience.
- Refined GPU detection logic and added error handling for missing NVIDIA drivers.
…er guidance
- Added WSL update command with error handling to ensure the latest version is used.
- Improved feedback messages for setting WSL2 as the default version, including instructions for manual updates if necessary.
Comment threadtracebloc/templates/_helpers.tpl
… experience
- Implemented asynchronous WSL update with progress feedback to enhance user guidance during installation.
- Improved logic for selecting a WSL2 distribution, prioritizing Ubuntu and providing fallback options.
- Updated commands to ensure compatibility and clarity in executing WSL commands within the script.
…8s.ps1
- Adjusted console encoding to properly handle WSL output, ensuring accurate retrieval of available distributions.
- Enhanced logic for selecting a WSL2 distribution, improving fallback behavior when no Ubuntu distro is found.
- Updated command execution for kubectl and helm to ensure compatibility and clearer output handling.
- Added logic to determine the real hardware architecture on macOS, ensuring the correct version of Docker Desktop is installed.
- Updated the installation script to download the appropriate Docker DMG based on the detected architecture.
- Implemented verification to warn users if the installed Docker binary does not match the hardware architecture.
- Added a spinner utility to provide visual feedback during long-running commands in the installation scripts.
- Updated Docker, kubectl, k3d, and helm installation processes to use the new spinner functionality, enhancing user experience by indicating progress.
- Refactored existing command execution to improve clarity and maintainability.
- Improved the waiting mechanism for Docker Desktop to include a visual spinner, providing real-time feedback while waiting for the Docker engine to start.
- Added informative messages to guide users if Docker is not responding, ensuring clarity on what to check during the startup process.
- Enhanced user experience by replacing static messages with dynamic updates during the waiting period.
…up script
- Added a new function to identify and terminate any lingering Docker Desktop processes that may block a clean startup.
- Integrated the cleanup function into the Docker installation process to ensure a smooth user experience during setup.
- Enhanced overall reliability of the installation script by preventing potential startup issues related to Docker.
…on improvements
- Introduced a new function to determine the Windows architecture, ensuring the correct binaries are downloaded for Docker Desktop and kubectl installations.
- Enhanced input validation for configuration parameters, including stricter checks for CLUSTER_NAME, SERVERS, AGENTS, HTTP_PORT, and HTTPS_PORT.
- Implemented checksum verification for downloaded binaries to ensure integrity and prevent tampering.
- Added a tool directory initialization function to streamline the installation process and ensure necessary paths are set correctly.
…alidation improvements
- Added a -NoReboot switch to the install-k8s.ps1 script to allow users to skip the reboot prompt after enabling Windows features.
- Updated K8S_VERSION default to a pinned version (v1.29.4-k3s1) for better stability, with clear instructions on using the latest version.
- Improved validation for HOST_DATA_DIR to ensure it is under USERPROFILE and not a system path, enhancing security.
- Refined cluster creation logic in cluster.sh to ensure exact cluster name matching and improved handling of existing clusters.
- Enhanced GPU plugin deployment scripts to download manifests to a temporary file for better integrity checks and error handling.

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Installer fails on first run resolving nonexistent directory
    • validate_config now resolves HOST_DATA_DIR via its canonicalized parent when the target directory does not yet exist, so first-run defaults like ~/.tracebloc no longer fail.

Create PR

Or push these changes by commenting:

@cursor push 919298949e
Preview (919298949e)
diff --git a/scripts/lib/common.sh b/scripts/lib/common.sh--- a/scripts/lib/common.sh+++ b/scripts/lib/common.sh@@ -200,7 +200,15 @@
# HOST_DATA_DIR must be under $HOME and must not be a system path (security)
local dir="$HOST_DATA_DIR"
[[ "$dir" != /* ]] && dir="$HOME/$dir"
- dir="$(cd -P "$dir" 2>/dev/null && pwd)" || true+ if [[ -d "$dir" ]]; then+ dir="$(cd -P "$dir" 2>/dev/null && pwd)" || true+ else+ local parent base+ parent="$(dirname "$dir")"+ base="$(basename "$dir")"+ parent="$(cd -P "$parent" 2>/dev/null && pwd)" || true+ [[ -n "$parent" ]] && dir="${parent}/${base}" || dir=""+ fi
[[ -z "$dir" ]] && error "HOST_DATA_DIR could not be resolved: $HOST_DATA_DIR"
case "$dir" in
/) error "HOST_DATA_DIR cannot be root (/)"
This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

Comment threadscripts/lib/common.sh Outdated
- Added TRACEBLOC_SKIP_REBOOT_PROMPT parameter to install-k8s.sh to allow skipping the reboot prompt after NVIDIA driver installation.
- Updated common.sh to provide specific feedback for Docker first-time setup, improving user guidance during installation.
- Enhanced setup-macos.sh to clarify Docker Desktop setup instructions and signal expected exit behavior for first-time installations.
Comment threadaks/templates/rbac-rols.yaml
saadqbal added 12 commits March 2, 2026 14:58
- Improved user guidance for first-time Docker Desktop setup, clarifying steps and expectations.
- Enhanced feedback during Docker startup with dynamic waiting messages and increased maximum wait time.
- Updated error handling to inform users if Docker does not start in the expected timeframe.
- Added architecture detection to ensure the correct version of Docker Desktop is installed based on the user's hardware.
- Implemented checks to remove existing Docker installations that do not match the detected architecture, improving compatibility for Apple Silicon users.
- Streamlined the installation process by consolidating architecture checks and installation conditions.
- Updated the hdiutil command to use the -nobrowse option for better handling of mounted volumes.
- Added xattr command to remove extended attributes from the Docker.app, ensuring a smoother installation experience.
- Updated the validation logic for HOST_DATA_DIR to resolve the parent directory first, improving error handling and ensuring the directory is correctly set even if the target does not exist initially.
- Enhanced error messages to provide clearer feedback when the parent directory cannot be resolved, improving user guidance during configuration.
- Added a new function to detect headless environments and install Colima as the Docker runtime when no GUI session is available.
- Enhanced the Docker installation process to check for existing installations and start the Colima Docker daemon if necessary.
- Updated messaging to inform users about the headless setup and provide guidance on potential issues with Docker daemon startup.
- Renamed the function for detecting GUI sessions to improve clarity and reliability.
- Updated the logic to check for GUI sessions using the ownership of /dev/console, enhancing detection accuracy for headless environments.
- Adjusted the Docker installation process to utilize the new function, ensuring proper handling of headless setups.
- Added a new function to check Docker Desktop architecture on macOS, providing clear instructions for users if the installed version does not match their hardware.
- Updated the Docker installation process to prompt users before replacing incorrect architecture installations, improving user experience and guidance.
- Enhanced messaging to inform users about potential performance issues related to architecture mismatches, ensuring better clarity during setup.
- Refactored the Docker binary path detection to use the correct main executable, improving accuracy in architecture validation.
- Enhanced the logic to check for the Docker binary's architecture, ensuring compatibility with both Intel and ARM architectures on macOS.
- Modified the Docker removal command to include 'sudo' for proper permissions, ensuring successful removal of Docker.app during installation.
- This change enhances the installation process by preventing potential permission issues when removing existing Docker installations.
- Added comments to clarify the need for 'sudo' when removing Docker.app due to protected paths.
- Noted that the official uninstall script is not used to avoid blocking issues during non-interactive runs, enhancing the installation process documentation.
- Updated echo statements in install.sh to provide clearer context about the Tracebloc client installer.
- Improved the banner in common.sh to reflect the Tracebloc branding and provide additional information about the setup process.
- Adjusted summary messages in summary.sh to emphasize the readiness of the Tracebloc client and the Kubernetes cluster, enhancing user feedback.
- Modified the zypper update command to remove the '-q' flag for improved output clarity during package refresh.
- Ensured the installation command retains the '-y' flag for non-interactive installations, maintaining user experience consistency.
Comment threadaks/templates/jobs-manager-deployment.yaml
- Added a fast-path check to see if k3d is already installed, improving efficiency.
- Implemented a fallback to use the openSUSE/SLES package manager (zypper) for installation when available, enhancing compatibility.
- Maintained checksum verification for downloaded k3d releases to ensure integrity, with improved error handling for checksum fetching.
- Updated success and warning messages for better user feedback during the installation process.
- Enhanced the fallback mechanism for k3d installation on openSUSE/SLES to log errors quietly when the package is not found.
- Updated warning messages for better clarity regarding the fallback to the upstream GitHub installer, improving user feedback during the installation process.
- Replaced the fallback mechanism for k3d installation with the official installer script for improved reliability and simplicity.
- Removed the previous zypper installation logic and checksum verification, streamlining the installation process.
- Added error handling to ensure k3d is available on PATH after installation, enhancing user feedback and installation success verification.

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment threadaks/templates/jobs-manager-deployment.yaml
- Introduced a new section in README.md detailing the installation of the tracebloc Helm chart for production environments.
- Created INSTALL.md to provide comprehensive installation instructions, including prerequisites, installation methods, required configurations, and upgrade/rollback procedures.
- Added a GitHub Actions workflow for packaging and publishing the tracebloc Helm chart to GitHub Pages, ensuring users can easily access the chart for installation.
- Removed outdated PVC identifiers from README.md for clarity and conciseness.
- Changed the 'minAvailable' setting to 'maxUnavailable' in jobs-manager-pdb.yaml to align with updated deployment strategies.
@saadqbal
saadqbal merged commit 9415680 into developMar 4, 2026
1 check passed
saadqbal added a commit that referenced this pull request Jun 19, 2026
Resolve conflict in scripts/tests/install-client-helm.bats. Both develop
(#262 / backend#743 — HOST_DATASET_DIR datasetPath + host uid/gid tests)
and this branch (#834 — TRACEBLOC_CLIENT_* non-interactive credential
tests) inserted new @test blocks at the same spot, so git interleaved
them. Kept all four tests as contiguous blocks.
scripts/lib/install-client-helm.sh auto-merged cleanly: #266 touches the
credential-gathering block, develop touches the values.yaml heredoc
(HOST_UID/GID + datasetPath) — non-overlapping. Verified the merged file
is the exact union of both parents.
install-client-helm.bats: 40/41 pass. The lone failure (#16
_extract_yaml_value single-quote escape) is byte-identical to develop and
fails only under macOS bash 3.2.57 — pre-existing and environmental, not
from this merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saadqbal added a commit that referenced this pull request Jun 19, 2026
) (#266)
RFC-0001 Phase 0 (no backend dependency):
- Accept TRACEBLOC_CLIENT_ID / TRACEBLOC_CLIENT_PASSWORD so CI / automation /
golden images can provision without typing the secret inline. Verified the
same way as the prompt (verify_credentials); a bad credential fails the
install (no re-prompt in non-interactive mode). The interactive prompt path
is unchanged — just wrapped in the else branch.
- Soften the "to connect this machine you NEED a tracebloc client / create
one" copy to "already have one? enter it (or set the env vars) / need one?
create it" — so we stop framing client-creation as a mandatory pre-step
(browser sign-in lands in Phase 1).
Tests: two new bats cases (env path -> non-interactive write + helm with no
prompt; rejected env creds -> error, no helm). The interactive flow tests
(re-prompt / inactive / unverified / defaults / max-attempts / one-client
guard) still pass unchanged.
NOTE: bats #16 (_extract_yaml_value single-quote '' un-escape) fails locally
on macOS bash — pre-existing and untouched by this PR (the diff doesn't go
near that function); flagging for confirmation against CI.
Part of backend#830. Closes#834.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Asad Iqbal <asad.dsoft@gmail.com>
saadqbal added a commit that referenced this pull request Jun 23, 2026
… backend-reachability test, installer hardening) (#273)
* Merge pull request #260 from tracebloc/fix/job-resource-defaults-745
test(jobs-manager): lock spawned-job RESOURCE_REQUESTS/LIMITS default at 8Gi
* test(e2e-proxy): exercise application-pod egress through the proxy (Charité setup) (#264)
* test(e2e-proxy): exercise application-pod egress through the proxy
The squid harness proved NODE egress (image pulls) but stopped before any
application pod — so it never caught client-runtime#119, where the spawned
ingestion Job carried no proxy env and dialled the backend directly. Add a
section that runs a pod WITH the ingestion-style proxy env (must traverse the
squid to reach the backend) and a pod WITHOUT it (must bypass it / go direct),
asserting both against the squid access log.
Models the Charité proxy-only setup at the application layer; pairs with the
behavioural unit tests on client-runtime#119.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(e2e-proxy): rework app-pod egress to an in-cluster squid Service
Running the first version on a real k3d cluster surfaced that a POD cannot
resolve host.k3d.internal (it is a node-level alias for image pulls, not pod
DNS), so the proxied probe failed with `curl (5) Could not resolve proxy`.
Rework: stand up an in-cluster squid Deployment+Service the test pods reach by
Service DNS (also a closer model of a real corporate proxy reachable by name),
with a readiness probe gating rollout on squid actually listening (fixes the
probe-before-bind race seen in the first attempt). A pod WITH the ingestion
proxy env must reach the backend through the squid; a pod WITHOUT it must bypass
it. Auth survival stays covered by the host-squid sections (1-3).
bash -n + shellcheck + embedded-YAML parse all clean; Service-DNS resolution
verified locally. Full proxied-curl run is exercised by the e2e-proxy CI job.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(e2e-proxy): make app-pod egress assertion deterministic (single pod, curl -v)
§4 now uses ONE pod carrying the ingestion-style proxy env that makes two
calls to the same backend: WITH the env it must tunnel via the in-cluster
squid (a CONNECT tunnel); with the env unset it must dial direct. Proof is
taken client-side from `curl -v` (the CONNECT-tunnel lines), not by reading
squid's access.log — that file is buffered by the log daemon and came back
empty when read right after the probe, producing false failures.
Also set BOTH proxy-env cases: curl honours the lower-case `https_proxy`
for HTTPS and the upper-case alone is not reliably picked up, so the probe
must emit both — exactly as the real ingestion env does. A single pod with
a single log also removes the multi-pod scheduling / log-flush races that
made the earlier two-pod form flaky.
Validated end-to-end on k3d:
A (proxy env) -> "Establish HTTP proxy tunnel to api.tracebloc.io:443"
+ "CONNECT tunnel established, response 200" + 200 OK
B (env unset) -> direct connect to the backend IP, no proxy tunnel, 200
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* ci(security): add public-repo PII gate caller (#263)
Blocks PRs that leak customer/partner names or secrets in title/body/commits.
Calls the reusable gate in tracebloc/.github. Inactive until the org
PII_DENYLIST secret is set (warns, doesn't block, until then).
* ci: add concurrency cancellation + job timeouts to chart CI (#265)
helm-ci.yaml and installer-tests.yaml are the repo's most expensive
workflows (a real k3d cluster, a 9-distro docker-in-docker matrix, Windows
Pester) but had no concurrency control, so a PR re-push left stale runs
burning to completion. Add a per-ref concurrency group that cancels
superseded PR runs only (push/schedule runs are never cancelled), matching
the pattern already used in client-runtime's tests.yml. Add timeout-minutes
to every job so a hung k3d/squid/distro step can't run to the 6h default.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(installer): fail fast when HOST_DATA_DIR is on a network filesystem (#261)
Detect NFS/CIFS/SMB for HOST_DATA_DIR in preflight (bash + PowerShell) and
fail fast with an actionable message instead of a cryptic MySQL
CrashLoopBackOff ~20 min into install: MySQL/InnoDB corrupts on network
storage and the chart root chown init-container is blocked by NFS root_squash.
- preflight.sh: _pf_fstype reader (findmnt, then GNU stat, then df+mount;
portable incl. macOS) + _pf_storage_type wired into run_preflight. Allowlists
network fstypes so local FSes including overlay/tmpfs (CI) pass.
- install-k8s.ps1: Get-PfFsType (UNC / network drive) + Test-Preflight check.
- TRACEBLOC_ALLOW_NETWORK_FS=1 overrides (mirrors TRACEBLOC_ALLOW_ARM64).
- Tests: 10 bats cases + Pester cases (network -> fail, override, undetermined,
Windows-only Get-PfFsType reader).
Part 1 of 3 for tracebloc/backend#743.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(installer,chart): place datasets on a network mount while MySQL stays local (#262)
* feat(installer): fail fast when HOST_DATA_DIR is on a network filesystem
Detect NFS/CIFS/SMB for HOST_DATA_DIR in preflight (bash + PowerShell) and
fail fast with an actionable message instead of a cryptic MySQL
CrashLoopBackOff ~20 min into install: MySQL/InnoDB corrupts on network
storage and the chart root chown init-container is blocked by NFS root_squash.
- preflight.sh: _pf_fstype reader (findmnt, then GNU stat, then df+mount;
portable incl. macOS) + _pf_storage_type wired into run_preflight. Allowlists
network fstypes so local FSes including overlay/tmpfs (CI) pass.
- install-k8s.ps1: Get-PfFsType (UNC / network drive) + Test-Preflight check.
- TRACEBLOC_ALLOW_NETWORK_FS=1 overrides (mirrors TRACEBLOC_ALLOW_ARM64).
- Tests: 10 bats cases + Pester cases (network -> fail, override, undetermined,
Windows-only Get-PfFsType reader).
Part 1 of 3 for tracebloc/backend#743.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(installer,chart): place datasets on a network mount while MySQL stays local
Storage split for VMs whose real storage is an NFS/CIFS mount (backend#743):
the database must stay on local disk (InnoDB over NFS is unsafe) but the large
dataset volume can live on the network mount.
Chart:
- Parameterize the dataset PV hostPath base via hostPath.datasetPath (helper
tracebloc.clientDataHostPath). Default /tracebloc keeps it byte-identical;
mysql + logs PV paths are unchanged. values.yaml + schema + nil-guard for
--reuse-values upgrades.
Installer (bash + PowerShell):
- New HOST_DATASET_DIR: validated (must exist + be writable; MAY live outside
$HOME unlike HOST_DATA_DIR; system paths barred), bind-mounted into k3d at a
distinct /tracebloc-data path; the dataset dir is created there while mysql +
logs stay local. When set, the generated values set
hostPath.datasetPath=/tracebloc-data and (Linux) pass HOST_UID/HOST_GID env to
jobs-manager so spawned ingestion pods write the host-owned NFS export as the
owning uid. Preflight notes the dataset dir is exempt from the network-FS block.
Tests: new shared_images_pvc_test.yaml + mysql/logs split-only guards
(helm-unittest, 259 pass); HOST_DATASET_DIR validation, second-mount, dir-split
and values-generation cases (bats). Docs: INSTALL.md checklist + SECURITY.md 5.4.
Part 2/3 of backend#743. The end-to-end NFS write path also needs the
client-runtime ingestor-uid change (separate PR) so jobs-manager reads HOST_UID.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#262): fail fast when an existing cluster lacks the dataset bind mount (Bugbot)
The HOST_DATASET_DIR -> /tracebloc-data bind mount is baked into the k3d nodes at
create time (_create_new_cluster / the PS1 equivalent). k3d cannot add a mount to a
RUNNING cluster, but install-client-helm.sh still wrote `datasetPath: /tracebloc-data`
into the generated values whenever HOST_DATASET_DIR was merely set — so an
existing-cluster re-run pointed the chart's dataset PV at ephemeral in-node storage,
silently putting datasets on disposable storage instead of the network export (lost
on a restart).
Add _check_existing_cluster_dataset_mount (cluster.sh) + the PowerShell equivalent,
mirroring the existing _check_existing_cluster_proxy/bind drift checks: on an existing
cluster with HOST_DATASET_DIR set, inspect the server node for the /tracebloc-data
mount and FAIL FAST with the recreate remedy if it is absent — rather than installing
a quietly misrouted dataset volume. Fail-fast (not warn) because this is silent data
loss, consistent with the network-FS fail-fast guard. Values generation needs no
change: the install now stops in Step 2, before helm runs.
+4 bats (cluster.bats): unset -> no-op, mount present -> pass, mount ABSENT -> fail
fast, inspect fails -> no-op. bash + shellcheck clean; pwsh parses the .ps1; full
cluster suite 27/27.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(installer): non-interactive credentials + softer connect copy (#834) (#266)
RFC-0001 Phase 0 (no backend dependency):
- Accept TRACEBLOC_CLIENT_ID / TRACEBLOC_CLIENT_PASSWORD so CI / automation /
golden images can provision without typing the secret inline. Verified the
same way as the prompt (verify_credentials); a bad credential fails the
install (no re-prompt in non-interactive mode). The interactive prompt path
is unchanged — just wrapped in the else branch.
- Soften the "to connect this machine you NEED a tracebloc client / create
one" copy to "already have one? enter it (or set the env vars) / need one?
create it" — so we stop framing client-creation as a mandatory pre-step
(browser sign-in lands in Phase 1).
Tests: two new bats cases (env path -> non-interactive write + helm with no
prompt; rejected env creds -> error, no helm). The interactive flow tests
(re-prompt / inactive / unverified / defaults / max-attempts / one-client
guard) still pass unchanged.
NOTE: bats #16 (_extract_yaml_value single-quote '' un-escape) fails locally
on macOS bash — pre-existing and untouched by this PR (the diff doesn't go
near that function); flagging for confirmation against CI.
Part of backend#830. Closes#834.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Asad Iqbal <asad.dsoft@gmail.com>
* test(e2e-proxy): deflake §4 app-egress — hermetic target + non-silent diagnostics (#269)
§4 ("APPLICATION-pod egress through a proxy", client-runtime#119) was a flaky
required check ("E2E auth-proxy (squid)") that intermittently red-X'd develop
(~1 in 4; e.g. run 27765964135) and randomly blocked unrelated PRs. Two causes:
1. Silent failure. Under `set -euo pipefail` the diagnostic `grep | sed` lines
ran before the real assertion; an empty section made grep exit 1 → pipefail
→ set -e killed the script with NO output (CI showed only "pod/egress-app
created" then "exit code 1"). Append `|| true` so the diagnostics are
non-fatal and the assertion fires with its reason. Same footgun fixed in §3.
2. External-network dependency (the real flake). §4 curled the real
https://api.tracebloc.io/ through the in-cluster squid, depending on the
runner's internet to a production host at test time. Make it hermetic: target
a reserved-TLD stand-in host (backend.tracebloc-e2e.test) aliased via
hostAliases on both the squid and app pods to the cluster's own kube-apiserver
ClusterIP — a guaranteed in-cluster HTTPS:443 listener. The CONNECT tunnel now
terminates in-cluster with zero external I/O, preserving the #119 intent
(WITH proxy env → CONNECT tunnel via squid; env unset → direct dial).
Validated: 3/3 deterministic local passes; both calls hit 10.43.0.1 in-cluster
(no api.tracebloc.io reachout). bash -n + shellcheck --severity=error clean.
Closes#268
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli#90): in-cluster backend-reachability helm test (WS3) (#270)
Adds `egress-reachability-check`, a `helm test` Job that verifies a normal
(non-training) pod in the namespace can reach the tracebloc backend API — the
egress dependency that gates everything (the cluster authenticates to the
backend to obtain its Service Bus credentials, so no backend egress => silent
Pending). The required-egress complement to egress-enforcement-check (which
verifies the opposite: that training pods are locked out).
The probe is deliberately NOT training-labelled (so the lockdown netpol never
selects it — it keeps the jobs-manager/requests-proxy egress class) and honours
tracebloc.proxyEnv, so it tests the real path. The verdict keys on curl's exit
code (TCP reachability), not HTTP status. Run via `helm test <release>`; gated
by egressReachabilityCheck.enabled (default true; disable on truly air-gapped
clusters). As a test hook it never runs during install/upgrade.
Service Bus is intentionally not probed here: its host is fetched post-auth
from the backend (static nowhere in the chart) and its egress is brokered by
the requests-proxy, whose readiness `tracebloc cluster doctor` already checks.
helm-unittest: 6 tests (render / disable / test-hook annotation / not-training-
labelled / CLIENT_ENV-driven host / proxy-inherited). Full chart suite green.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(chart): bump client 1.7.1 → 1.8.0 (version + appVersion) (#272)
Release v1.8.0 — promotes the 9 commits merged to develop since v1.7.1.
Minor bump: #262 (datasets on a network mount while MySQL stays local)
is a real PVC-placement change at install time, not inert. Keeps
version/appVersion in lockstep so the app.kubernetes.io/version label
matches the release.
Refs #271
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(egress-reachability-check): fail closed on TLS/transport errors, not just connect failures (#274)
The backend-reachability helm test (#270) keyed success on a denylist of
curl exit codes (5/6/7/28) and passed on everything else. But with no
--fail, curl exits 0 for any HTTP response, so the only non-zero codes are
transport/TLS failures — yet a TLS handshake/cert error (e.g. 35/51/60,
typically a proxy intercepting TLS with a CA the cluster doesn't trust)
printed "backend reachable" and exited 0: a false pass that defeats the
check's purpose.
Pass ONLY on curl exit 0 (a full TCP+TLS+HTTP round trip proves the
backend is reachable and usable); fail closed on everything else, with a
dedicated TLS/cert bucket and a generic catch-all. Companion helm-unittest
asserts nothing on the script body; full suite 265/265.
Follow-up to #270; flagged by Cursor Bugbot on the v1.8.0 sync PR (#273).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@saadqbal