Skip to content

feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157

Merged
drewstone merged 1 commit into
mainfrom
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged

feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom
feat/enterpriseops-gym-adapter

Conversation

@drewstone

@drewstonedrewstone commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.

@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone merged commit d1d26d3 into mainJun 4, 2026
1 check passed
@drewstone
drewstone deleted the feat/enterpriseops-gym-adapter branch June 4, 2026 21:56
@drewstonedrewstone mentioned this pull request Jun 6, 2026
Merged
drewstone added a commit that referenced this pull request Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) by drewstone · Pull Request #157 · tangle-network/agent-runtime · GitHub
Skip to content
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain) - #157
Merged
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Jun 4, 2026
Merged
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker (the gate's non-coding middle-band domain)#157
drewstone merged 1 commit into
mainfrom 
feat/enterpriseops-gym-adapter
Conversation
@drewstone
@drewstonedrewstone
commented
Jun 4, 2026

Copy link


Copy Markdown

Contributor
Wires EnterpriseOps-Gym (ServiceNow-AI) as a real BenchmarkAdapter through the unifier. It's the gate's ideal domain: a deployable programmatic verifier — the agent's tool-call transcript is replayed against a freshly-seeded gym MCP server, then the task's own SQL state-checks run via /api/sql-runner (NOT an answer-oracle, NOT an LLM-judge). Graded per-verifier (score = passes/total; resolved = all-pass), covering the enterprise-ops user story. loadTasks pulls the HF dataset with a committed offline fixture; judge fails loud with the exact docker pull/run/seed step when the gym server is unreachable — never a fabricated score. One line in adapters.ts. Verified: bench tsc 0, fixture test 5/5, lint clean. Note: dataset card Apache-2.0 vs paper CC-BY-NC-SA — confirm before redistribution.
@drewstone
feat(bench): EnterpriseOps-Gym adapter — deployable SQL state-checker
820ac34
The gate's ideal NON-coding domain: a deployable programmatic verifier (the
agent's tool-call transcript is replayed against a freshly-seeded gym MCP server,
then the task's own SQL state-checks run via /api/sql-runner) — NOT an answer-
oracle. Graded per-verifier (score = passes/total = the bench's
verifier_level_pass_rate; resolved = all-pass = overall_success_rate). Covers the
enterprise-ops user story (itsm/hr/csm/calendar/email/drive/teams).
Real adapter on the existing pattern: loadTasks pulls the HF dataset
(ServiceNow-AI/EnterpriseOps-Gym) with a committed offline fixtures fallback;
judge invokes the live-env driver and FAILS LOUD with the exact docker pull/run/
seed step when the gym server is unreachable — never a fabricated score.
Registered in adapters.ts (one line). goldArtifact undefined (the oracle is the
seeded DB state, not a portable transcript — documented, not faked).
Note: dataset card says Apache-2.0; the paper lists CC-BY-NC-SA — confirm before
redistribution. Verified: bench tsc 0, fixture test 5/5, repo lint clean.
@drewstone
drewstone
merged commit d1d26d3
into
mainJun 4, 2026

1 check passed

@drewstone
drewstone
deleted the

feat/enterpriseops-gym-adapter

branch
June 4, 2026 21:56
@drewstonedrewstone
mentioned this pull request
Jun 6, 2026
 Merged

drewstone added a commit
that referenced
this pull request
Jun 6, 2026
@drewstone
chore(release): agent-runtime 0.45.0 (#176)
86306e4
Cuts the 58-commit backlog on main into a published release. Headline surface:
- runToolLoop / streamToolLoop — bounded turn-level tool-dispatch loop (#137)
- RSI agent tree: recursive Agent.act, Supervisor keystone, runProgram, the
adaptive-driver channel (#139/#151/#165)
- optimization API collapsed onto agent-eval selfImprove; the runtime keeps the
CODE-surface ImprovementDriver you pass as driver (#172)
- deployable benchmark adapters: AppWorld, commit0, aec-bench, EnterpriseOps-Gym;
runBenchmarks over one ADAPTERS registry (#153/#156/#157)
- agent-eval floor raised to >=0.83.0 (#175)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone