chore(examples): clearer names — drop confusing with- prefix - #206

Merged
drewstone merged 8 commits into
mainfrom
chore/examples-naming
Jun 9, 2026
Merged

chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom
chore/examples-naming

Conversation

@drewstone

@drewstonedrewstone commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).

oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches

Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).

git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.

@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone merged commit 29aba34 into mainJun 9, 2026
1 check passed
@drewstone
drewstone deleted the chore/examples-naming branch June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
chore(examples): clearer names — drop confusing with- prefix - #206
Merged
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Jun 9, 2026
Merged
chore(examples): clearer names — drop confusing with- prefix#206
drewstone merged 8 commits into
mainfrom 
chore/examples-naming
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 9, 2026

Copy link


Copy Markdown

Contributor
Disciplined subset of the examples-naming audit. Renames the genuinely-confusing names; rejects the audit's 01-08 numbering + .deprecated/ quarantine (churn for throwaway examples; the README already orders them).
oldnewwhy
with-knowledge-readinessknowledge-gatingwith- reads as an optional toggle, not a primary pattern
with-intelligence-exportintelligence-exportsame
agent-into-reviewerpipe-into-reviewersignals the 2-runtime piping it teaches
Kept (audit wanted to rename, I pushed back): runtime-run (it teaches startRuntimeRun — the name matches the product API; renaming would disconnect it), agents-of-all-shapes (memorable + has a test).
git mv preserves history; examples/README.md + docs/concepts.md + all internal self-references updated; zero stragglers.
@drewstone
feat(bench): GEPA over the analyst/steerer prompt on the canonical stack
1041c54
The analyst IS the steerer (observe()'s findings → recommended_action → the depth
steer), so optimizing the analyst prompt optimizes the loop. This evolves it with
agent-eval's REAL GEPA primitives (buildReflectionPrompt + parseReflectionResponse
+ paretoFrontier) — no hand-rolled optimizer; there is no turnkey runPromptEvolution
in agent-eval 0.83, only the primitives, so the population loop is thin orchestration
over them.
- observe(): + analystInstruction? override (the analyst prompt is now the GEPA knob);
defaultAnalystInstruction exported. Firewall stays structural (input has no score).
- agentic.ts: AgenticOptions.analystInstruction threads into the depth steerer.
- eops-gepa.mts: FITNESS = depth-vs-breadth lift on the canonical Supervisor+observe
gate; breadth computed ONCE per task (shared baseline, correct + halves cost);
failing per-task lifts = the reflection gradient. Seeds = observe()'s PROVEN default
(the +16.4pp instruction) FIRST, then the designer-panel population.
Smoke (N=2, 1 gen) validated the full loop: score → paretoFrontier select → reflect
→ mutate → re-score → pick. Bounded real run (N=6, 2 gens) in flight.
@drewstone
fix(bench): GEPA harness survives gym/router infra blips (skip failed…
ea4845f
… tasks)
The first real run died when the (long-lived) gym container wedged: breadth
baselines returned 0% then runAgentic threw 'every rollout went down', killing the
whole GEPA run. runAgentic is fail-loud; the GEPA loop now catches per-task: a task
whose rollouts fail is SKIPPED (not fatal), both in the breadth precompute and the
depth fitness. Fails loud only if <2 tasks survive (genuine infra-down). Pair with a
fresh gym container + WIDTH<=2.
@drewstone
refactor(bench): delete eops-gate.mts — the throwaway flat-loop proto…
dfca540
…type (−433 LOC)
It was a dead-end (nothing imports it): a hand-rolled flat loop that BYPASSED the
canonical Supervisor + a second copy of the gym client (6 functions duplicating
gym-agent.ts's 5). Fully superseded by the canonical stack — agentic.ts (domain-blind
depth/breadth/Supervisor/observe, 428 LOC, written ONCE) + the AgenticSurface seam
(agentic-eops.ts, 73 LOC = the entire per-domain slot-in). The +16.4pp result and the
GEPA harness run on the canonical path; this prototype only de-risked the plumbing
(gym standup, router-tools worker, depth-best scoring) and is now dead weight.
@drewstone
feat(bench): package the optimization suite (runBenchmark) + clarify …
6f3c15d
…naming + onboarding fixes
The pieces existed (Supervisor + observe + the depth/breadth strategies) but weren't
wrapped as a usable suite, and the vocabulary was opaque. runBenchmark is the packaged
front door:
runBenchmark({ environment, tasks, worker, strategies: ['sample','refine'], budget })
→ runs each strategy, scores by the environment's own deployable check, returns the
per-strategy means + the paired-bootstrap lift of refine over sample. printBenchmarkReport
gives the verdict. Resilient to transient per-task infra (skip, don't crash).
Naming, made legible (public API; maps to internal depth/breadth — zero churn to the
running internals): a task domain is an `Environment` (the AgenticSurface seam under the
RL/gym-standard name); the strategies are `sample` (best-of-N / resample) and `refine`
(attempt → critic reads trace → steer → repeat), named by what they DO, not the search
tree's shape. Juniors call runBenchmark; seniors customize the hooks (worker.analystInstruction
= the critic, Environment.score = the check) or drop to runAgentic for new strategies.
Onboarding: deleted the orphaned empty examples/define-loop/ (defineLoop removed #194);
fixed the dead examples/model-resolution link in docs/concepts.md.
@drewstone
feat(bench): make Strategy a first-class, OPEN abstraction (author yo…
e52b8d5
…ur own)
The question: when we collapse to "refine", can a dev create their OWN strategy?
Before: no — runAgentic took mode:'depth'|'breadth', a CLOSED enum. The capability
existed (a strategy is an Agent) but the door wasn't cut.
Now: `Strategy` is an exported interface — `{ name, driver(surface, task, opts, budget)
=> Agent }`. A strategy builds the driver Agent the Supervisor runs; author your own by
returning an Agent whose act() spawns shots/analysts via scope.spawn/next/send. `refine`
and `sample` ship as instances AND the reference driver implementations (depthDriver/
breadthDriver) are exported to copy. runAgentic accepts a `strategy` (mode kept for
back-compat); runBenchmark takes `Strategy[]` — pass the built-ins or your own.
What's under the words:
sample = K independent attempts, keep the best-verifying (best-of-N / resample)
refine = attempt → observe() reads the trace → steer the next → repeat (iterate)
A multi-agent "team" is just a Strategy whose driver spawns several different agents —
same recursive Agent atom, coordinated over the Scope.
@drewstone
feat(bench): defineStrategy + composable steps — author a loop in ~15…
1dfbfd6
… lines (skillifiable)
The original goal: loops compact enough to skillify, so agents author them. A 70-line
Supervisor driver isn't that. This adds the composable LEGO:
defineStrategy(name, async ({ shot, critique, surface, budget }) => { ...compose... })
A strategy body gets two steps — shot() (one worker attempt over an artifact) and
critique() (the firewalled analyst reads the trace → a steer) — with ZERO Supervisor/
Scope/spawn/leaf/drainOne ceremony (all of it lives inside defineStrategy now). That is
the unit an agent or a skill can emit.
Proof: adaptiveRefine — a NEW strategy (refine, but ABANDON-and-restart when a steered
shot fails to improve = branch-when-stuck, the widen/MCTS idea the depth-stuck failure
motivated), authored entirely from the steps, scored keep-best. ~22 lines of pure
strategy logic, no plumbing.
Behavior-preserving: the proven refine/sample drivers (depthDriver/breadthDriver) are
UNTOUCHED — the +16.4pp result + GEPA stay valid. The steps replicate their exact
spawn/drain pattern, so a step-authored strategy behaves identically. Typecheck-verified;
adaptiveRefine live-smoke pending the gym (GEPA has it).
@drewstone
docs(bench): strategy-demo example — the optimization suite in 3 laye…
ab13798
…rs (gym-free, runnable)
The missing onboarding piece: a runnable demo of the whole suite on a toy "counter"
Environment (needs only a router key — no dataset, no sandbox). Shows all three layers:
1. runBenchmark(env, …) — default strategies compared, free.
2. strategies: [sample, refine, adaptiveRefine] — pick, named by behavior.
3. defineStrategy('doubleCheck', body) — author your own in ~10 lines from shot()+critique(),
zero Supervisor ceremony. The skillifiable unit.
Verified: runs end-to-end through the canonical Supervisor; all 4 strategies execute and
score via the Environment's own check. README documents the model + the customization hooks.
@drewstone
chore(examples): clearer names — drop the confusing with- prefix; c…
658160b
…larify intent
Disciplined subset of the examples-naming audit (NOT the proposed 01-08 numbering /
.deprecated quarantine — that's churn for throwaway examples and the README already
orders them):
with-knowledge-readiness → knowledge-gating (`with-` read as an optional toggle)
with-intelligence-export → intelligence-export (same)
agent-into-reviewer → pipe-into-reviewer (signals the 2-runtime piping)
KEPT runtime-run (it teaches startRuntimeRun — the name matches the product API) and
agents-of-all-shapes (memorable + has a test). git mv preserves history; README +
docs/concepts + all internal self-references updated; zero stragglers.
@drewstone
drewstone
merged commit 29aba34
into
mainJun 9, 2026

1 check passed

@drewstone
drewstone
deleted the

chore/examples-naming

branch
June 9, 2026 12:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone