docs: rewrite README around a Linux-first setup path

README.md

@@ -1,80 +1,125 @@
 # intermesh
 
-## Contributing
+Intermesh is a trust-based identity mesh written in Rust. Nodes exchange signed
+control statements about names, addresses, and policy. Each node evaluates that
+state locally to build its own view of the mesh. By bringing control to the
+node, Intermesh distributes responsibility for trust across the network.
+
+Intermesh runs on Linux. Build and test on Linux to match production and CI.
+
+## Getting Started
+
+### Platform
+
+- Linux is required for Intermesh build and test workflows.
+- Use a Linux host directly (bare metal, VM, or cloud instance), or use
+ Docker Desktop/Colima and run your workflow in a Linux container (mount the
+ repo and the Docker socket).
+
+### Prompt-Driven Setup (especially useful on macOS/Windows)
+
+If you are on macOS or Windows and want an assistant to drive setup, paste this
+into Claude Code (or similar).
+
+Budget 10-20 minutes (sometimes a touch longer) for a first run. Most of that
+is unattended downloads and a cold Rust build. Several steps print nothing for
+minutes at a time; that is normal, not a hang.
+
+```text
+I need a Linux-based Intermesh dev environment on this machine.
+Create the smallest reliable setup and explain each step briefly.
+
+Requirements:
+- Intermesh tooling must run in Linux
+- Docker daemon must be available
+- Read this Intermesh README (where this prompt came from) first and treat it
+ as the source of truth
+- Repo is already cloned at <PATH_TO_INTERMESH> (ask user if no path was provided)
+- Prefer manual tool install in Linux (rustup + protoc + cmake + ruby +
+ pkg-config + musl-tools + git + build-essential)
+- If using a container: mount the repo and the Docker socket, and set
+ DOCKER_BUILDKIT=1 (the e2e image build requires BuildKit)
+- If using a container: ensure docker CLI, buildx, and docker compose are
+ installed in that container
+- Set LANG=C.UTF-8 and LC_ALL=C.UTF-8 before running cargo local
+- Nix is optional, not required
+
+Please:
+1) Detect host OS and choose a Linux execution path (native Linux, or Linux container/VM).
+2) Give exact commands to set up dependencies.
+3) Run verification commands and show pass/fail:
+ - docker info
+ - cargo --version
+ - cargo local
+4) Warn me before any step that takes more than a few minutes, and say how
+ long it should take. The first cargo local runs 10-20 minutes and goes
+ silent for long stretches; do not report it as hung or kill it.
+5) If a step fails, show likely cause and next command to run.
+6) End with a short checklist of what is done vs what is still needed.
+```
 
 ### Docker
 
-You need Docker installed and running. On Linux, install it via your
-distribution's package manager. On macOS, you can use
-[Docker Desktop](https://www.docker.com/products/docker-desktop/) or
-[Colima](https://github.com/abiosoft/colima). Development on macOS works
-but is not as heavily tested as Linux — if you hit mac-specific issues,
-ask in Slack.
+Docker is required for e2e tests. Install it with your distro package manager
+on Linux, then confirm:
 
-The e2e tests use a shared Nix-built Docker base image. xtask builds a static
-musl intermesh binary for the host architecture and copies it into a thin image
-layered on that base, so Rust code changes do not invalidate the slower tools
-layer.
+```bash
+docker info
+```
 
-### Build Dependencies
+## Setup
 
-#### Option 1: Nix + direnv (Recommended)
+### Option 1: Manual Setup (recommended)
 
-[Nix](https://nixos.org/) is a package manager focused on reproducible
-builds that works on Linux and macOS. You don't need to understand it —
-it will just make sure you get all the right dependencies at the right
-versions, guaranteed to be compatible with each other. If you're curious,
-[Zero to Nix](https://zero-to-nix.com/) is a good introduction.
+If you want a straightforward, roll-your-own setup, install:
 
-- Install Nix — the [Determinate Systems installer](https://github.com/DeterminateSystems/nix-installer)
- is the easiest option, though some prefer the
- [official installer](https://nixos.org/download/)
-- Install [direnv](https://direnv.net/) and hook it into your shell
- (bonus points if you use `nix profile install nixpkgs#direnv`)
+- `rustup` (then run `rustup toolchain install` in this repo)
+- `protoc`
+- `cmake`
+- `ruby`
+- `pkg-config`
+- `musl-tools`
+- `git`
+- a C toolchain (`build-essential` on Debian/Ubuntu)
+- `flock` (usually provided by `util-linux`)
 
-Once both are installed, `cd` into the repo and run `direnv allow`. From
-that point on, every time you enter the directory the dev shell provides
-system build tools plus the Rust toolchain selected by
-`rust-toolchain.toml`, including clippy, rustfmt, rust-analyzer, and the
-musl target used by e2e tests. If the Rust version changes, re-enter the
-directory or run `direnv reload`.
+`rust-toolchain.toml` pins Rust, clippy, rustfmt, rust-analyzer, and targets.
 
-#### Option 2: Manual Installation
+### Option 2: Nix + direnv
 
-If you'd rather manage dependencies yourself, you'll need:
+Nix gives a reproducible dev shell with the exact toolchain and versions:
 
-- [rustup](https://rustup.rs/)
- - After installing it, run `rustup toolchain install` from the repo root
- - Rustup reads `rust-toolchain.toml`, which selects Rust, clippy,
- rustfmt, rust-analyzer, and the musl target
-- protobuf compiler (`protoc`)
-- cmake
-- Ruby (for the custom linter)
-- pkg-config
+- Install [Nix](https://nixos.org/) (the
+ [Determinate installer](https://github.com/DeterminateSystems/nix-installer)
+ is the easiest path)
+- Install [direnv](https://direnv.net/) and hook it into your shell
+- From repo root: `direnv allow`
 
-This is mostly standard Rust ecosystem tooling. You're on your own for
-version management, but it shouldn't be too hard.
+If you'd rather not manage tool versions yourself, use this.
 
-### Verify Your Setup
+## Verify Your Setup
 
-A good first goal is to get `cargo local` to pass:
+Run the full local check:
 
 ```bash
 cargo local
 ```
 
-This runs formatting, linting, a full build, unit tests, and e2e tests.
-If it passes, you're mostly good to go.
+This runs formatting, lint, build, unit tests, and e2e tests.
+
+Expect 10-20 minutes the first time. The cold Rust build and the e2e image
+build take most of the time, and both go quiet for minutes at a time. Later
+runs are much faster once the build cache and the e2e base image are warm.
 
-## Development Commands
+## Core Commands
 
-- `cargo build` — Build the project
-- `cargo test [filter]` — Run unit tests (use the filter to run a specific
-test while iterating, e.g. `cargo test my_test_name`)
-- `cargo lint` — Run format check, clippy, and custom lints
-- `cargo local` — Lint, build, unit tests, and e2e tests
-- `cargo all` — Everything in `local` plus VM tests when Incus is available
-- `cargo deps` — Check for outdated/unused/vulnerable dependencies
+- `cargo build` - build the project
+- `cargo test [filter]` - run unit tests; pass a filter to run just the tests
+whose name matches it, e.g. `cargo test my_test_name`
+- `cargo lint` - run format check, clippy, and custom lints
+- `cargo local` - lint + build + unit tests + e2e tests
+- `cargo all` - `local` plus perf tests and VM tests when Incus is available
+- `cargo deps` - dependency freshness and vulnerability checks
 
 ## E2E Tests
 
@@ -83,54 +128,77 @@ cargo test-e2e # run all e2e tests
 cargo test-e2e test_name # run a specific test
 ```
 
-The e2e tests spin up Docker containers on a local bridge network. Each
-container uses the shared Nix-based test image plus the intermesh binary
-built from your working tree. The only requirement beyond the build
-dependencies is a running Docker daemon.
+E2E tests run Docker containers on a local bridge network with binaries built
+from your working tree.
 
-## VM Tests
+If you run inside a dev container, install `docker` CLI + buildx + compose in
+that container so commands like `cargo test-e2e` and `docker compose` work.
 
-The VM tests exercise intermesh across isolated local Incus guests.
-
-### Setup
+## VM Tests
 
-You do not need Incus for normal local development. The VM integration tests
-run in CI, and most contributors are not expected to have Incus set up locally.
-You only need a working local Incus installation if you want to run those tests
-before pushing. The Nix dev shell includes the `incus` CLI, but it does not
-install or configure the host Incus daemon for you.
+VM tests run intermesh across isolated local [Incus](https://linuxcontainers.org/incus/)
+VMs. They run in CI, so most contributors never need a local Incus setup;
+reach for them when you want guest-level network realism before pushing.
 
-The harness does not require a NixOS-specific Incus setup. It needs a local
-Incus daemon that your user can access, VM support enabled, and a default
-profile with a root disk and network. The host must be able to launch the
-Ubuntu 24.04 cloud VM image and the guest must have outbound network access
-while the cached image is built:
+You only need this if you plan to run VM tests locally. The Nix shell includes
+the `incus` CLI, but it does not install or configure the host Incus daemon.
+Your host should be able to launch the Ubuntu 24.04 cloud VM image:
 
 ```bash
 incus image info images:ubuntu/24.04/cloud --vm
 incus launch images:ubuntu/24.04/cloud intermesh-incus-check --vm --ephemeral
 incus delete --force intermesh-incus-check
 ```
 
-The VM image build and tests currently request 4 CPUs and 4 GiB of memory per
-VM. No specific storage driver is required; our NixOS machines use a ZFS-backed
-default pool, but any Incus storage pool that can run VMs should work.
-
-`cargo test-vm` runs the VM integration tests in ephemeral Incus VMs, building
-a reusable local test image on first use.
-
-### Running
+Quick commands:
 
 ```bash
-cargo xtask vm-image # ensure the cached VM image exists and print its alias
+cargo xtask vm-image # ensure the cached VM image exists, print its alias
 cargo test-vm # run all VM tests; fails if Incus is unavailable
 cargo test-vm test_name # run a specific test
-cargo all # run everything; skips VM tests if Incus is unavailable
 cargo xtask clean-vm-images # drop cached local Intermesh VM images
 ```
 
-Unlike e2e tests, the smoke test runs `intermesh` directly on the guest host
-so it can operate on the VM's real network interfaces. Test workloads run in
-that guest's local Docker daemon. `cargo all` attempts the VM suite but skips
-it when Incus is unavailable; use `cargo test-vm` when VM coverage must be
-enforced.
+`cargo all` attempts VM tests and skips them when Incus is unavailable.
+
+## Local Mesh Demo
+
+The repo ships a small local mesh demo in `sandbox/`:
+
+```bash
+# first run takes time
+cd sandbox && docker compose up -d
+```
+
+See `sandbox/README.md` for what to try once it's up and running.
+
+## Day-to-Day Workflow
+
+A tight loop that works well:
+
+1. `cargo test [filter]`
+2. `cargo lint`
+3. `cargo test-e2e [filter]`
+4. `cargo local` before push
+
+## Common Gotchas
+
+- **Docker daemon not reachable:** start Docker and re-run `docker info`
+- **Lint fails with `invalid byte sequence in US-ASCII`:** set
+ `LANG=C.UTF-8 LC_ALL=C.UTF-8`
+- **First e2e run is slow:** expected; the e2e base image and build cache have
+ to warm up. Long silent stretches are normal, so let it finish
+- **VM tests skipped:** expected without a local Incus setup
+- **Using a Linux dev container on macOS/Windows:** set
+ `CARGO_TARGET_DIR` to a container-local path to avoid host/guest target dir
+ conflicts (for example `export CARGO_TARGET_DIR=/tmp/intermesh-target`)
+
+Stuck on something platform-specific? File a GitHub issue:
+<https://github.com/pcnofelt/intermesh/issues>
+
+## Next Steps
+
+1. Get `cargo local` passing
+2. Run `cargo test-e2e` once to warm the environment
+3. Bring up the `sandbox/` demo mesh
+4. Pick a small issue and follow the day-to-day loop above