Skip to content

建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80

Merged
mindcarver merged 2 commits into
mainfrom
feat/74-current-ab-benchmark
Aug 30, 2026
Merged

建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom
feat/74-current-ab-benchmark

Conversation

@mindcarver

@mindcarvermindcarver commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Refs #74

结果

  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。

真实 smoke

  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830

这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。

验证

  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED

风险与边界

  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。

@mindcarvermindcarver mentioned this pull request Aug 30, 2026
Closed
14 tasks
@mindcarver
mindcarver merged commit d873a9e into mainAug 30, 2026
1 check passed
@mindcarver
mindcarver deleted the feat/74-current-ab-benchmark branch August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
建立 Thinloop 当前版本三臂整体价值 A/B 评测 by mindcarver · Pull Request #80 · mindcarver/thinloop · GitHub
Skip to content
建立 Thinloop 当前版本三臂整体价值 A/B 评测 - #80
Merged
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Aug 30, 2026
Merged
建立 Thinloop 当前版本三臂整体价值 A/B 评测#80
mindcarver merged 2 commits into
mainfrom 
feat/74-current-ab-benchmark
Conversation
@mindcarver
@mindcarvermindcarver
commented
Aug 30, 2026

Copy link


Copy Markdown

Owner
Refs #74
结果
  • 新增 evals/thinloop/ 当前版本三臂整体评测：native、固定短提示、当前十二个 Thinloop Skill。
  • 版本化覆盖清晰 Bug、定义不足功能、多 Issue、中断恢复、脏工作树、浏览器验收和伪完成七类任务。
  • 提供隔离 fixture / Codex Home、脱敏与秘密扫描、公开确定性评分、指标聚合、不可覆盖 Run ID、保存观察与无模型重评分。
  • dry 无认证/无模型；smoke 固定最小三臂真实用例；full 浏览器证据缺失时失败关闭为 BLOCKED。
  • 普通 CI 只接入 definitions 和 dry，保留 建立 Thinloop 路由内核与防漂移校验 #75 的 routing-kernel check，不调用付费模型。
真实 smoke
  • Run：issue-74-smoke-f075dfe-20260830
  • Source：f075dfebe94b362b166627ee01694fa9b68c897f，运行时工作树干净
  • Codex CLI 0.143.0，gpt-5.4，medium reasoning
  • native / prompt / thinloop：均完成真实 turn，隐藏单引号行为与仓库测试均 PASS
  • 整轮：OBSERVED；秘密扫描 0；无模型 rescore 仍为 OBSERVED
  • 证据：/Users/carver/workspace/mindcarver/.worktrees/test/thinloop-current-eval/runs/issue-74-smoke-f075dfe-20260830
这只证明三个条件在一个固定伪完成 fixture 上都通过；没有观察到相对增益，不宣称统计显著性、效果百分比或整体价值。未提供明确模型单价，成本保持不可确认。
验证
  • node --test tests/*.test.mjs：187/187 PASS
  • node scripts/sync-routing-kernel.mjs --check：PASS
  • Discovery / Knowledge / current Thinloop definitions：PASS
  • Knowledge dry 与 current Thinloop dry：PASS
  • node scripts/generate-readme-diagrams.mjs --check：PASS
  • claude plugin validate . --strict：PASS
  • node evals/thinloop/runner/rescore.mjs --run <run>：OBSERVED
风险与边界
  • 评测工具、测试、CI 接入和文档，不修改生产 Skill 行为。
  • 页面任务仍要求外部真实浏览器证据；普通 CI 和本次最小 smoke 不伪造该证据。
  • 其余六类真实 subject、重复样本和其他模型尚未运行，已在 EVALUATION.md 明确列为限制。
@mindcarvermindcarver
mentioned this pull request
Aug 30, 2026

Closed

14 tasks
@mindcarver
mindcarver
merged commit d873a9e
into
mainAug 30, 2026

1 check passed

@mindcarver
mindcarver
deleted the

feat/74-current-ab-benchmark

branch
August 30, 2026 08:49
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@mindcarver