Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); feat(intelligence): ship reviewable improvement lifecycle by drewstone · Pull Request #511 · tangle-network/agent-runtime · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -68,13 +68,15 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.
`improve` optimizes one part of an agent and **only ships a change if it beats the current agent on tasks it never practiced on**.
It accepts prompt, skill document, tool, MCP, hook, subagent, workflow, rollout-policy, whole-profile, and code surfaces through one call.
Prompt and skill-document optimization have built-in generators; structured surfaces take an explicit generator, and code runs from isolated incumbent and candidate checkouts.

```ts
import { improve } from '@tangle-network/agent-runtime'

const { profile, shipped, lift } = await improve(baseProfile, findings, {
surface: 'prompt', // what to optimize: prompt | skills | code
surface: 'prompt', // or skills/tools/mcp/hooks/subagents/workflow/agent-profile/code
gate: 'holdout', // certified on a held-back exam, never the practice set
scenarios, judge, agent, // how to measure a candidate
})
Expand Down
1 change: 1 addition & 0 deletions docs/api/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,7 @@

- [agent](agent.md)
- [analyst-loop](analyst-loop.md)
- [candidate-execution](candidate-execution.md)
- [index](index.md)
- [intelligence](intelligence.md)
- [knowledge](knowledge.md)
Expand Down
90 changes: 77 additions & 13 deletions docs/api/agent.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -865,7 +865,7 @@ Required for `mode === 'open-pr'` — the GH owner/repo (`tangle-network/tax-age

##### allowCreateForKinds?

> `optional` **allowCreateForKinds?**: readonly (`"memory"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]
> `optional` **allowCreateForKinds?**: readonly (`"code"` \| `"mcp"` \| `"memory"` \| `"workflow"` \| `"agent-profile"` \| `"rollout-policy"` \| `"skill"` \| `"hook"` \| `"subagent"` \| `"knowledge.wiki"` \| `"knowledge.claim"` \| `"knowledge.raw"` \| `"knowledge.stale"` \| `"system-prompt"` \| `"tool-doc"` \| `"new-tool"` \| `"rag"` \| `"scaffolding"` \| `"output-schema"` \| `"websearch.outdated"` \| `"prior-run-summary"` \| `"cluster"`)[]

Defined in: [agent/improvement-adapter.ts:100](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/improvement-adapter.ts#L100)

Expand DownExpand Up@@ -1544,51 +1544,115 @@ Defined in: [agent/surfaces.ts:55](https://github.com/tangle-network/agent-runti

Optional: single file defining the output schema (Zod / JSON Schema).

##### skills?

> `optional` **skills?**: `string`

Defined in: [agent/surfaces.ts:57](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L57)

Optional: directory containing Agent Skill packages.

##### mcp?

> `optional` **mcp?**: `string`

Defined in: [agent/surfaces.ts:59](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L59)

Optional: directory containing MCP server/tool configuration.

##### hooks?

> `optional` **hooks?**: `string`

Defined in: [agent/surfaces.ts:61](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L61)

Optional: directory containing hook definitions.

##### subagents?

> `optional` **subagents?**: `string`

Defined in: [agent/surfaces.ts:63](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L63)

Optional: directory containing subagent definitions.

##### workflows?

> `optional` **workflows?**: `string`

Defined in: [agent/surfaces.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L65)

Optional: directory containing orchestration/workflow policies.

##### rolloutPolicy?

> `optional` **rolloutPolicy?**: `string`

Defined in: [agent/surfaces.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L67)

Optional: single file containing rollout-policy settings.

##### agentProfile?

> `optional` **agentProfile?**: `string`

Defined in: [agent/surfaces.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L69)

Optional: single canonical AgentProfile file.

##### code?

> `optional` **code?**: `string`

Defined in: [agent/surfaces.ts:71](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L71)

Optional: source root for code findings.

***

### ResolvedSurface

Defined in: [agent/surfaces.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L58)
Defined in: [agent/surfaces.ts:74](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L74)

#### Properties

##### absolutePath

> **absolutePath**: `string`

Defined in: [agent/surfaces.ts:60](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L60)
Defined in: [agent/surfaces.ts:76](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L76)

Absolute filesystem path the operator can `cat` / `vim`.

##### repoRelativePath

> **repoRelativePath**: `string`

Defined in: [agent/surfaces.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L62)
Defined in: [agent/surfaces.ts:78](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L78)

Repo-relative path for PR descriptions, diffs, audit logs.

##### exists

> **exists**: `boolean`

Defined in: [agent/surfaces.ts:64](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L64)
Defined in: [agent/surfaces.ts:80](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L80)

Whether the path currently exists on disk.

##### intent

> **intent**: `"edit-existing"` \| `"create-new"`

Defined in: [agent/surfaces.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L66)
Defined in: [agent/surfaces.ts:82](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L82)

The substrate's intent: edit an existing file or create a new one.

***

### SurfaceValidationIssue

Defined in: [agent/surfaces.ts:191](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L191)
Defined in: [agent/surfaces.ts:264](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L264)

Validate that every declared surface exists on disk under `repoRoot`.

Expand All@@ -1603,19 +1667,19 @@ the loop produces 20 minutes later).

> **surface**: keyof [`AgentSurfaces`](#agentsurfaces)

Defined in: [agent/surfaces.ts:192](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L192)
Defined in: [agent/surfaces.ts:265](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L265)

##### path

> **path**: `string`

Defined in: [agent/surfaces.ts:193](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L193)
Defined in: [agent/surfaces.ts:266](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L266)

##### reason

> **reason**: `"missing"` \| `"not-directory"` \| `"not-file"`

Defined in: [agent/surfaces.ts:194](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L194)
Defined in: [agent/surfaces.ts:267](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L267)

## Type Aliases

Expand DownExpand Up@@ -1993,7 +2057,7 @@ resolves only after the iterator drains.

> **resolveSubjectPath**(`subject`, `surfaces`, `repoRoot`): [`ResolvedSurface`](#resolvedsurface) \| `null`

Defined in: [agent/surfaces.ts:86](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L86)
Defined in: [agent/surfaces.ts:102](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L102)

Resolve a parsed `FindingSubject` to the file path the substrate
should edit (or create) on disk.
Expand DownExpand Up@@ -2035,7 +2099,7 @@ it's the whole point.

> **validateSurfaces**(`surfaces`, `repoRoot`): readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

Defined in: [agent/surfaces.ts:198](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L198)
Defined in: [agent/surfaces.ts:271](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L271)

Validate an `AgentSurfaces` map on disk — missing paths fail loud at `defineAgent` time instead of silently skipping self-improvement edits.

Expand All@@ -2059,7 +2123,7 @@ readonly [`SurfaceValidationIssue`](#surfacevalidationissue)[]

> **renderSurfaceIssues**(`issues`, `repoRoot`): `string`

Defined in: [agent/surfaces.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L247)
Defined in: [agent/surfaces.ts:345](https://github.com/tangle-network/agent-runtime/blob/main/src/agent/surfaces.ts#L345)

Format a list of surface validation issues into a human-readable error string.

Expand Down
Loading
Loading