From 21d6f7be83406d673f8d8748678382dd51cf7a7b Mon Sep 17 00:00:00 2001 From: kristo Date: Fri, 19 Jun 2026 19:56:48 +0300 Subject: [PATCH] docs: prepare main-device handoff --- README.md | 6 +- docs/handoff.md | 67 ++++++++++++++----- ...26-06-01-real-sandbox-red-team-commands.md | 16 +++-- docs/progress.md | 14 ++-- docs/runbook.md | 2 +- 5 files changed, 74 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 4f4a03e..6e1bc3d 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ The V1 demo proves public cloud-storage exposure in local sandboxes, applies Ter 8. Validate the attack path is blocked. 9. Write case-study-ready evidence and metrics. -Hackathon V1 includes live LocalStack runs for AWS S3 and Azure Blob scenarios, plus offline deterministic demos for Kubernetes, Docker Compose, on-prem baselines, and generic plan review. +Hackathon V1 includes a live-validated LocalStack AWS S3 path, a manifest-backed Azure Blob probe path that still needs live LocalStack Azure entitlement to validate, plus offline deterministic demos for Kubernetes, Docker Compose, on-prem baselines, and generic plan review. ## Why this exists @@ -280,7 +280,7 @@ Users do not need to write prompts. `nullstate` sends internal red-team and blue | Scenario | Backend | Status | |---|---|---| -| `azure-public-blob` | `localstack-azure` | live LocalStack demo available | +| `azure-public-blob` | `localstack-azure` | manifest-backed/offline demo available; live LocalStack Azure validation pending emulator entitlement | | `aws-public-s3` | `localstack-aws` | live LocalStack demo available | | `k8s-privileged-pod` | `kind-kubernetes` | offline demo available; live kind pending | | `compose-exposed-admin` | `docker-compose` | offline demo available; live Docker probe pending | @@ -430,6 +430,6 @@ cosign verify-blob $wheel.FullName --bundle "$($wheel.FullName).sigstore.json" - ## Status -Working now: live LocalStack AWS/Azure storage scenarios, offline deterministic demos for all listed scenarios, constrained red attack command execution, deterministic remediation, sandbox registry, report artifacts, model metrics artifacts, branded CLI output, and DevSecOps repo structure. +Working now: live LocalStack AWS storage validation, manifest-backed Azure Blob probe generation pending live LocalStack Azure entitlement, offline deterministic demos for all listed scenarios, constrained red attack command execution, deterministic remediation, sandbox registry, report artifacts, model metrics artifacts, branded CLI output, and DevSecOps repo structure. Experimental: richer scenario-specific attack scripts, live Kubernetes/Compose/on-prem adapters, richer resolved-dependency SBOMs, and broader artifact redaction coverage. diff --git a/docs/handoff.md b/docs/handoff.md index 0ad304e..3e68e8c 100644 --- a/docs/handoff.md +++ b/docs/handoff.md @@ -1,6 +1,6 @@ # Nullstate Project Handoff -Last updated: 2026-06-18 +Last updated: 2026-06-19 ## Read this first @@ -14,6 +14,7 @@ The hackathon freeze rule was lifted on 2026-06-18 by the project owner. PR #24 Recent merged checkpoint: ```text +f621b1b docs: record PR 24 merge 1e7b1c6 feat: add constrained red attack runner ``` @@ -26,6 +27,39 @@ git log --oneline -8 Do not rely on MCP state, chat memory, or remote PR metadata. Use local files and Git only unless the user explicitly provides other tooling. +## Fresh agent checklist + +When this project is opened on the main device: + +1. Confirm branch and cleanliness: + + ```powershell + git status --short --branch + git log --oneline -8 + ``` + +2. Confirm local secret files are present only locally and ignored by Git: + + ```powershell + git check-ignore .env .env.local + ``` + +3. Recreate the local Python environment if needed: + + ```powershell + python -m pip install -e . + ``` + +4. Run the verification gate before taking new work: + + ```powershell + python -m ruff check src tests + python -m mypy src + python -m unittest discover -s tests -v + ``` + +5. Start a new feature branch for follow-up product work. Keep `main` protected and use PR checks. + ## Project goal `nullstate` is becoming an open-core DevSecOps product: @@ -154,7 +188,11 @@ docs/plans/2026-06-01-real-sandbox-red-team-commands.md ## Last verification run -The last full verification passed: +The last full verification passed on 2026-06-19 after reinstalling the project with dev extras: + +```powershell +python -m pip install -e .[dev] +``` ```powershell python -m ruff check src tests @@ -167,15 +205,15 @@ Result: ```text Ruff passed mypy passed -69 tests OK +146 tests OK ``` -Smoke run also passed: +Smoke run also passed on 2026-06-19: ```powershell -python -m nullstate run examples/aws-public-s3 --offline --mock-agents --runs-dir runs/platform-smoke -python -m nullstate bundle --runs-dir runs/platform-smoke -python -m nullstate dashboard --runs-dir runs/platform-smoke +python -m nullstate run examples/aws-public-s3 --offline --mock-agents --runs-dir runs/handback-smoke-20260619-195333 +python -m nullstate bundle --runs-dir runs/handback-smoke-20260619-195333 +python -m nullstate dashboard --runs-dir runs/handback-smoke-20260619-195333 ``` Generated: @@ -238,7 +276,7 @@ Recommended next tests: - Live LocalStack Azure run where available. - Offline run still passes. - Report classifies runtime evidence as observed/inconclusive/simulated. -- Remaining enterprise hardening: future live-cloud approval gate and live Azure emulator validation after LocalStack Azure entitlement is available. +- Remaining enterprise hardening: live Azure emulator validation after LocalStack Azure entitlement is available. If Azure LocalStack support is unavailable or unreliable, do not overclaim Azure runtime exploitation. Prefer clear report language such as `runtime probe inconclusive; deterministic IaC validation still blocked the configured exposure`. @@ -259,7 +297,7 @@ Run bundle is the key contract between: - support tickets - future enterprise dashboards -Current productization checkpoints on `feature/red-agent-runner` also include: +Current productization checkpoints merged into `main` include: - provider presets for Google, Claude, custom, and generic OpenAI-compatible endpoints - SARIF export and GitHub Actions code-scanning workflow @@ -300,11 +338,10 @@ Do not jump straight to full SaaS before stabilizing: ## Branch and release guidance -Until the user says the freeze is over: +The hackathon freeze is over, but `main` remains protected: -- Do not merge PR #24. -- Do not push or merge to `main`. -- Feature branch checkpoint pushes are allowed. +- Start substantive work on a feature branch. +- Open PRs into `main` and wait for required checks. +- Direct pushes to `main` are blocked by branch protection. - After applying CodeRabbit review feedback, do not push solely to trigger another CodeRabbit review loop; fold those fixes into the next substantive batch unless the user explicitly approves a review-response push. -- Do not tag releases. -- Do not update `main`. +- Do not tag releases or publish product releases without explicit owner approval. diff --git a/docs/plans/2026-06-01-real-sandbox-red-team-commands.md b/docs/plans/2026-06-01-real-sandbox-red-team-commands.md index 689c9ec..f3d7fea 100644 --- a/docs/plans/2026-06-01-real-sandbox-red-team-commands.md +++ b/docs/plans/2026-06-01-real-sandbox-red-team-commands.md @@ -2,6 +2,8 @@ > **For Agent:** Use executing-plans skill to implement this plan task-by-task. +> **Status as of 2026-06-19:** This plan is mostly implemented and was merged into `main` through PR #24. Do not replay Tasks 1-6 from scratch. Use this file as historical context and continue only with the remaining live LocalStack Azure validation work after the account has Azure Emulator entitlement. + **Goal:** Replace shallow LocalStack health probes with scenario-specific red-team probes that attempt real sandbox reads before and after remediation while preserving strict command boundaries. **Architecture:** Keep the existing constrained `attack.py` runner. Move exploit logic into scenario templates that receive a target URL, stage, and a generated evidence manifest. The deterministic detector remains the source of truth, but the report distinguishes between configuration validation and real runtime exploit evidence. @@ -12,18 +14,19 @@ ## Current State -The current red-team execution feature is safe but shallow: +The current red-team execution feature is safe and partly runtime-backed: - `src/nullstate/attack_runner.py` executes only generated `attack.py` inside the run directory. - `events.jsonl` records command, stdout, stderr, return code, target URL, stage, timestamps, and duration. -- AWS/Azure `attack.py` scripts currently call `/_localstack/health` when online. -- The before/after `success` and `blocked` verdict still comes from `simulate_attack()`. +- AWS has a live-validated LocalStack object-read probe path. +- Azure has a manifest-backed blob-read probe path, but live LocalStack Azure validation is still blocked by Azure Emulator entitlement. +- The before/after final verdict still relies on deterministic IaC validation for reliability; reports distinguish observed, inconclusive, and simulated runtime evidence. This is a strong security boundary, but not yet a full enterprise exploit validation engine. ## Progress Status -Updated 2026-06-09: +Updated 2026-06-19: - Completed locally: Task 1 attack evidence manifest. - Completed locally: Task 2 safe runner manifest argument. @@ -58,9 +61,8 @@ Updated 2026-06-09: - Google AI Studio / Gemini users can provide only `NULLSTATE_LLM_PROVIDER=google` and `NULLSTATE_LLM_API_KEY` - Claude is routed through Anthropic's OpenAI SDK compatibility endpoint and documented as experimental - self-hosted AMD/vLLM/SGLang users can still provide explicit shared or role-specific base URLs -- Verified locally with Ruff, mypy, full unittest discovery, and an offline Azure smoke run. -- Not pushed: local feature-branch work should remain local unless the user explicitly asks to push. -- Freeze rule: do not merge to `main`, do not push unless the user explicitly asks. +- Merged into `main` through PR #24 after review blockers and required checks passed. +- Verified locally on 2026-06-19 with Ruff, mypy, full unittest discovery, and an offline AWS handback smoke run. Next task to execute: diff --git a/docs/progress.md b/docs/progress.md index 4fddf7c..1e80bcd 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -66,17 +66,17 @@ This file tracks roadmap progress after the hackathon freeze. Keep updates brief - Added `nullstate policy validate`. - Writes optional `policy-validation.json` for CI evidence. - Exits with code `2` when the policy file is malformed or invalid. -- `fix: harden policy and evidence verification` (local checkpoint) +- `fix: harden policy and evidence verification` (merged in PR #24) - Reviewed the latest CodeRabbit findings and accepted the verified fail-closed issues. - `nullstate policy-result` now fails closed when `findings.json` is missing, malformed, or not a list. - `nullstate evidence-verify` now fails copied/wrong manifests whose declared run identity does not match the target run. - Malformed evidence manifests now surface as CLI parameter errors instead of Python tracebacks. - - This checkpoint is intentionally local until the next substantive batch push, to avoid triggering a CodeRabbit review loop over review-response fixes. -- `fix: make SARIF upload and CodeQL fixtures CI-clean` (local checkpoint) + - Batched with the next productization push to avoid a CodeRabbit review loop over review-response fixes. +- `fix: make SARIF upload and CodeQL fixtures CI-clean` (merged in PR #24) - Investigated PR #24 failing checks with `gh`. - Fixed GitHub SARIF upload validation by moving finding logical locations under SARIF result locations. - Reworked scrubber test fixture values/naming so CodeQL does not treat the test artifact as clear-text secret storage. - - GitHub has not re-run these checks yet because the fixes remain local on this branch. + - GitHub checks later passed on PR #24 before merge. - `feat: add scenario policy presets` (this checkpoint) - Added `nullstate policy init --scenario `. - Scenario presets narrow generated `allowed_scenarios` and `allowed_backends` to one known scenario/backend pair. @@ -136,7 +136,7 @@ This file tracks roadmap progress after the hackathon freeze. Keep updates brief - `docs: add first release verification checklist` (this checkpoint) - Added a runbook checklist for the first tagged release. - Captures PR check review, manual release dry-run rehearsal, release inspection, GitHub attestation verification, and Sigstore bundle verification. - - Keeps the freeze rule explicit: no tagging, release publishing, or `main` updates until approved. + - Keeps release approval explicit: no tagging or release publishing until approved. - `feat: add versioned remediation metadata` (this checkpoint) - Added `remediation.json` with remediation ruleset version, scenario, changed files, and applied deterministic rule IDs. - Included remediation metadata in reports, run bundles, and blue-team remediation events. @@ -190,6 +190,10 @@ This file tracks roadmap progress after the hackathon freeze. Keep updates brief - `feat: add constrained red attack runner` (merged PR #24) - Squash-merged the productization branch into `main` after all review threads and remote checks were green. - Merge commit: `1e7b1c6`. +- `docs: prepare main-device handoff` (this checkpoint) + - Refreshed handoff, runbook, README, progress, and active plan docs for the post-freeze `main` state. + - Corrected Azure runtime wording so it does not overclaim live LocalStack Azure validation before emulator entitlement is available. + - Reinstalled local dev extras and reran Ruff, mypy, 146 unit tests, and an offline AWS handback smoke run. ## In Progress diff --git a/docs/runbook.md b/docs/runbook.md index d3342a7..d16bbfa 100644 --- a/docs/runbook.md +++ b/docs/runbook.md @@ -121,7 +121,7 @@ python -m nullstate scrub 20260509-200601 --runs-dir runs --output-dir scrubbed- ## First Tagged Release Checklist -Use this checklist before the first product tag. Do not tag, publish a release, merge to `main`, or push to `main` until the freeze is explicitly lifted. +Use this checklist before the first product tag. The hackathon merge freeze was lifted on 2026-06-18, but product tags and release publishing still need explicit owner approval. Keep normal changes on feature branches and merge through protected PR checks. Confirm the release candidate PR checks are green: