Add Kernel-backed Harbor control arm

CHANGELOG.md

@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 ### Added
+- Added Kernel as a managed remote browser runtime with live view and downloaded replay recordings.
 - Added support for the [WebBrain](https://github.com/webbrain-one/webbrain) harness. Thanks to @alectimison-maker.
 
 ## [0.9.1] - 2026-08-04

README.md

@@ -296,7 +296,7 @@ Once the container starts, the script prints a **noVNC URL** (e.g. `http://local
 clawbench-batch --models your-model --cases-suite v2 --all-cases
 ```
 
-`your-model` is a key you configured in step 1; `--cases-suite v2` runs the full V2 corpus (swap in `v1-lite` for the 20-task subset). Add `--max-concurrent N` to run tasks in parallel (default 2 locally, 1 with Browserbase) and `--harness <name>` to pick an agent. Each task is intercepted and scored by the `deepseek-v4-pro` judge from step 1 — pass `--no-judge` to skip scoring. A `batch-summary.json` plus per-run recordings land under `./test-output/`.
+`your-model` is a key you configured in step 1; `--cases-suite v2` runs the full V2 corpus (swap in `v1-lite` for the 20-task subset). Add `--max-concurrent N` to run tasks in parallel (default 2 locally, 1 with Kernel or Browserbase) and `--harness <name>` to pick an agent. Each task is intercepted and scored by the `deepseek-v4-pro` judge from step 1 — pass `--no-judge` to skip scoring. A `batch-summary.json` plus per-run recordings land under `./test-output/`.
 
 **By hand, to produce a human reference run:**
 
@@ -330,7 +330,7 @@ Full registry: [`src/clawbench/runtime/harnesses/harnesses.yaml`](src/clawbench/
 
 | I want to… | Where |
 | --- | --- |
-| Use a managed remote browser instead of a local container | [`docs/browser-runtimes.md`](docs/browser-runtimes.md) — Browserbase setup, options, recording URLs |
+| Use a managed remote browser instead of a local container | [`docs/browser-runtimes.md`](docs/browser-runtimes.md) — Kernel and Browserbase setup, options, and recordings |
 | Run V2 through the Harbor framework (and run it fast) | [`docs/harbor.md`](docs/harbor.md) — conversion, judge wiring, concurrency, troubleshooting |
 | See every CLI command and flag | [`docs/cli.md`](docs/cli.md) |
 
@@ -709,7 +709,7 @@ Each session records five layers of synchronized data under `/data/`:
 
 | Layer | File | Description |
 | ------------------ | ---------------------- | --------------------------------------------------------------- |
-| Session replay | `recording.mp4` or `run-meta.json` recording URL | Local H.264 video or Browserbase Session Inspector replay |
+| Session replay | `recording.mp4` or `run-meta.json` recording URL | Local/Kernel H.264 video or Browserbase Session Inspector replay |
 | Action screenshots | `screenshots/*.png` | Throttled timestamped PNGs captured after browser actions |
 | Browser actions | `actions.jsonl` | Every DOM event (click, keydown, input, pageLoad, scroll, etc.) |
 | HTTP traffic | `requests.jsonl` | Every HTTP request with headers, body, and query params |

docs/README.zh-CN.md

@@ -243,7 +243,7 @@ uv run clawbench-run test-cases/v1/001-daily-life-food-uber-eats claude-sonnet-4
 clawbench-batch --models your-model --cases-suite v2 --all-cases
 ```
 
-`your-model` 是你在第 1 步里配置的 key；`--cases-suite v2` 跑完整 V2 语料（换成 `v1-lite` 则是 20 题子集）。`--max-concurrent N` 控制并发（本地默认 2，Browserbase 默认 1），`--harness <name>` 选择智能体。每个任务都会被拦截并由第 1 步配置的 `deepseek-v4-pro` judge 打分 —— 加 `--no-judge` 可跳过评分。`batch-summary.json` 和各次运行的录制都会写到 `./test-output/`。
+`your-model` 是你在第 1 步里配置的 key；`--cases-suite v2` 跑完整 V2 语料（换成 `v1-lite` 则是 20 题子集）。`--max-concurrent N` 控制并发（本地默认 2，Kernel 或 Browserbase 默认 1），`--harness <name>` 选择智能体。每个任务都会被拦截并由第 1 步配置的 `deepseek-v4-pro` judge 打分 —— 加 `--no-judge` 可跳过评分。`batch-summary.json` 和各次运行的录制都会写到 `./test-output/`。
 
 **自己上手操作，产出人工参考轨迹：**
 
@@ -277,7 +277,7 @@ harness 是驱动浏览器的智能体框架，和模型是两个独立维度。
 
 | 我想…… | 去哪看 |
 | --- | --- |
-| 用托管的远程浏览器代替本地容器 | [`docs/browser-runtimes.md`](browser-runtimes.md) —— Browserbase 配置、参数、录制地址 |
+| 用托管的远程浏览器代替本地容器 | [`docs/browser-runtimes.md`](browser-runtimes.md) —— Kernel 和 Browserbase 配置、参数与录制 |
 | 用 Harbor 框架跑 V2（并且跑得快） | [`docs/harbor.md`](harbor.md) —— 转换、judge 配置、并发、排错 |
 | 查所有 CLI 命令和参数 | [`docs/cli.md`](cli.md) |
 
@@ -649,7 +649,7 @@ ClawBench 的定位：**真实消费级网站、日常任务、端到端录制**
 
 | 层 | 文件 | 描述 |
 |-------|------|-------------|
-| 会话回放 | `recording.mp4` 或 `run-meta.json` 中的录制 URL | 本地 H.264 视频，或 Browserbase Session Inspector 回放 |
+| 会话回放 | `recording.mp4` 或 `run-meta.json` 中的录制 URL | 本地/Kernel H.264 视频，或 Browserbase Session Inspector 回放 |
 | 动作截图 | `screenshots/*.png` | 浏览器动作后经限流捕获的带时间戳 PNG |
 | 浏览器动作 | `actions.jsonl` | 每个 DOM 事件 (click, keydown, input, pageLoad, scroll 等) |
 | HTTP 流量 | `requests.jsonl` | 每个 HTTP 请求，包含 headers、body 和查询参数 |

docs/browser-runtimes.md

@@ -2,12 +2,42 @@
 
 By default ClawBench launches Chromium inside its own container. You can point it at a managed remote browser instead — useful when the host cannot run containers comfortably, or when you want the provider to handle scaling and session replay.
 
-`--browser-runtime` accepts `local` (default), `browserbase`, `remote-cdp`, and `steel`. **`steel` is reserved and not implemented yet** — selecting it raises an error.
+`--browser-runtime` accepts `local` (default), `kernel`, `browserbase`, `remote-cdp`, and `steel`. **`steel` is reserved and not implemented yet** — selecting it raises an error.
 
 ## Local container (default)
 
 Nothing to configure. Chromium, Xvfb, ffmpeg, noVNC, and the recorder/interceptor all run in the task container; the session video lands at `recording.mp4`.
 
+## Kernel
+
+Put the key in `.env.local`:
+
+```dotenv
+KERNEL_API_KEY=...
+```
+
+Then select the runtime on a single or batch run:
+
+```bash
+uv run clawbench-run test-cases/v1/<case> your-model \
+ --browser-runtime kernel
+
+uv run clawbench-batch --models your-model --all-cases \
+ --browser-runtime kernel
+```
+
+Kernel runs use ClawBench's existing CDP action capture, screenshots, HTTP logging, and request interception. ClawBench starts a Kernel replay with each browser and downloads the completed video to `data/recording.mp4` before deleting the session.
+
+Provider options are passed as JSON. Supported fields are `stealth`, `region`, `proxy`, and `tags`:
+
+```bash
+uv run clawbench-batch --models your-model --all-cases \
+ --browser-runtime kernel \
+ --browser-runtime-options '{"stealth":true,"region":"us-east"}'
+```
+
+Set `KERNEL_BASE_URL` to override the default `https://api.onkernel.com` API endpoint. Batch concurrency defaults to **1**; raise `--max-concurrent` only as far as your Kernel account limit allows.
+
 ## Browserbase
 
 Put the key in `.env.local`:

docs/cli.md

@@ -42,7 +42,7 @@ clawbench-run <case-dir> --human # human reference run
 | `--output-dir <path>` | `<project>/test-output` | Where run directories are written |
 | `--no-build` | off | Skip building the container image (assumes it exists) |
 | `--no-upload` | off | Skip HuggingFace upload even if `HF_TOKEN` is configured |
-| `--browser-runtime <name>` | `local` | `local`, `browserbase`, `remote-cdp` — see [`browser-runtimes.md`](browser-runtimes.md) |
+| `--browser-runtime <name>` | `local` | `local`, `kernel`, `browserbase`, `remote-cdp` — see [`browser-runtimes.md`](browser-runtimes.md) |
 | `--browser-cdp-url <url>` | — | CDP endpoint for `--browser-runtime remote-cdp` |
 | `--browser-runtime-options <json>` | — | Provider-specific options, e.g. `'{"region":"us-west-2"}'` |
 
@@ -71,7 +71,7 @@ Execution:
 
 | Flag | Default | What it does |
 | --- | --- | --- |
-| `--max-concurrent <n>` | 2 local, 1 Browserbase | Parallel jobs |
+| `--max-concurrent <n>` | 2 local, 1 Kernel/Browserbase | Parallel jobs |
 | `--stagger-delay <s>` | 15 | Minimum seconds between consecutive container starts (rolling start) |
 | `--resume <dir>` | — | Reuse a previous batch's output directory and skip finished runs |
 | `--dry-run` | off | Print the job matrix without running anything |
@@ -125,6 +125,8 @@ See [Reproduce the leaderboard](../README.md#reproduce-the-leaderboard) for the
 | `CONTAINER_ENGINE` | Force `docker` or `podman` |
 | `HF_TOKEN` | Optional upload of runs to HuggingFace |
 | `BROWSERBASE_API_KEY` | Browserbase runtime (from `.env.local`) |
+| `KERNEL_API_KEY` | Kernel runtime (from `.env.local`) |
+| `KERNEL_BASE_URL` | Optional Kernel API base URL override |
 | `CLAWBENCH_JUDGE_*` | Judge credentials for Harbor's verifier — see [`harbor.md`](harbor.md) |
 
 PurelyMail credentials for disposable run emails come from the committed `.env`.

docs/harbor.md

@@ -108,6 +108,25 @@ uvx --from harbor==0.15.0 harbor run \
  --jobs-dir ./harbor-jobs/hermes-deepseek-flash
 ```
 
+## Kernel browser runtime (control arm)
+
+By default each Harbor trial runs Chromium inside its own container. Pass `--browser-runtime kernel` to the adapter to run the same tasks against one Kernel cloud browser per task instead:
+
+```bash
+uv run clawbench-harbor-adapt \
+ --output-dir ./harbor-datasets/clawbench-v2-kernel \
+ --browser-runtime kernel \
+ --browser-runtime-options '{"stealth": true}' \
+ --task-ids v2-1134-chapter-finder-redcross \
+ --overwrite
+```
+
+During task setup, the environment creates exactly one Kernel browser and replay, starts the ClawBench runtime server against it, and exposes only the local credential-free CDP bridge (`http://127.0.0.1:7878`) to the agent — the Kernel API key is never visible to the benchmark agent. Session identity and cleanup metadata land in `/my-info/kernel_browser.json`. During verification the provider replay is finalized, `recording.mp4` is downloaded into `/data`, and the browser is deleted (idempotently, including failure paths via a setup trap).
+
+Generated tasks register a pinned Playwright MCP package (`@playwright/mcp@0.0.79`) pointed at the CDP bridge, so Harbor's stock Claude Code and Codex agents drive the Kernel browser with native Playwright MCP tool calls — structurally identical to ClawBench's native Claude/Codex harnesses.
+
+Export `KERNEL_API_KEY` (and optionally `KERNEL_BASE_URL` for non-production gateways) before `harbor run`; no extra flags are needed.
+
 ## Making it fast
 
 A full V2 sweep is 129 containerized browser sessions, each capped by the task's `time_limit`. Serial, that is a very long night. What actually moves the needle, in order:

src/README.md

@@ -196,22 +196,22 @@ A stagger delay is applied between job starts since during container startup it
 | `--cases-dir PATH` | Custom case directory | none |
 | `--all-cases` | Use all task directories in the selected suite/dir | false |
 | `--case-range START-END` | Filter by numeric case ID prefix | none |
-| `--max-concurrent N` | Max parallel jobs; Browserbase defaults to 1 | 2 locally, 1 with Browserbase |
+| `--max-concurrent N` | Max parallel jobs; managed runtimes default to 1 | 2 locally, 1 with Kernel/Browserbase |
 | `--output-dir PATH` | Base output directory | `test-output` |
 | `--stagger-delay SECONDS` | Minimum gap between consecutive container starts | 15 |
 | `--dry-run` | Print job matrix without running | false |
 | `--no-upload` | Skip HuggingFace upload for all runs | false |
 | `--harness NAME` | Harness image to use | `openclaw` |
-| `--browser-runtime NAME` | Browser runtime (`local`, `remote-cdp`, or `browserbase`) | `local` |
+| `--browser-runtime NAME` | Browser runtime (`local`, `remote-cdp`, `kernel`, or `browserbase`) | `local` |
 | `--browser-cdp-url URL` | CDP endpoint used with `--browser-runtime remote-cdp` | none |
 | `--browser-runtime-options JSON` | Provider options such as Browserbase region or proxies | none |
 
-Browserbase reads `BROWSERBASE_API_KEY` from `.env.local` or the process
-environment. Its provider-hosted Session Inspector recording URL is stored in
-`browser_runtime.recording_url` in `run-meta.json`; local MP4 recording is
-thus omitted to save space and bandwidth. The signed provider CDP URL is mounted into the runtime
-container through a temporary read-only secret file and redacted from saved
-metadata.
+Kernel reads `KERNEL_API_KEY`; Browserbase reads `BROWSERBASE_API_KEY`. Both
+can be set in `.env.local` or the process environment. Kernel replay video is
+downloaded to `data/recording.mp4`; Browserbase's provider-hosted Session
+Inspector URL is stored in `browser_runtime.recording_url` in `run-meta.json`.
+Signed provider CDP URLs are mounted into the runtime container through a
+temporary read-only secret file and redacted from saved metadata.
 
 Signal handling: