Use shared workflow for JS+PY release - #204

Merged
David Elner (delner) merged 1 commit into
mainfrom
ci/release_shared_action
Jul 24, 2026
Merged

Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom
ci/release_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jul 15, 2026•
edited
Loading

Copy link
Copy Markdown
Collaborator

Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.

What changes

  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.

Benefits

  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.

@delnerDavid Elner (delner) added the enhancement New feature or request label Jul 15, 2026
@delnerDavid Elner (delner) mentioned this pull request Jul 15, 2026
Closed
@github-actions

github-actionsBot commented Jul 15, 2026•
edited
Loading

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1784904665)

ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴

@delner
David Elner (delner)force-pushed the ci/release_shared_action branch 3 times, most recently from 429951e to 2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner) marked this pull request as ready for review July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad) approved these changes Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?

@delnerDavid Elner (delner)Jul 23, 2026

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.

@delnerDavid Elner (delner) mentioned this pull request Jul 17, 2026
Merged
@delner
David Elner (delner) merged commit b0e1055 into mainJul 24, 2026
30 checks passed
@github-actions

github-actionsBot commented Jul 24, 2026•
edited
Loading

Copy link
Copy Markdown

Braintrust eval report

Autoevals (main-1784905235)

ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Use shared workflow for JS+PY release - #204
Merged
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Jul 24, 2026
Merged
Use shared workflow for JS+PY release#204
David Elner (delner) merged 1 commit into
mainfrom 
ci/release_shared_action
Conversation
@delner
@delnerDavid Elner (delner)
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Replaces the separate publish-js.yaml / publish-py.yaml workflows and their dispatcher with a single publish.yaml that releases the npm and PyPI autoevals packages together, at one shared version, behind one approval gate . The workflow is built on the shared release actions from braintrustdata/sdk-actions.
What changes
  • One workflow, two lanes (js + py): configure → validate → pack per lane, one shared request-approval, then a gated ship-package per lane.
  • Release actions perform attestation, SBOM and gating.
Benefits
  • One approval, both packages. A single reviewer approval releases npm and PyPI; if either lane
    
fails before the gate, neither ships.
  • Supply-chain hardening. An unprivileged job builds + generates a CycloneDX SBOM + attests (no publish creds); a gated job verifies the attestation and publishes those exact bytes via OIDC trusted publishing, npm provenance + PyPI PEP 740, no tokens.
  • Less bespoke YAML. Release logic lives in versioned, SHA-pinned shared actions; upstream improvements arrive with a SHA bump instead of hand-edits.
@delnerDavid Elner (delner)
added
the 
enhancement
New feature or request
label
Jul 15, 2026
@delnerDavid Elner (delner)
mentioned this pull request
Jul 15, 2026
 Closed

@github-actions
github-actionsBot
commented
Jul 15, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (HEAD-1784904665)
ScoreAverageImprovementsRegressions
NumericDiff79% (0pp)7 🟢9 🔴
Time_to_first_token9.8tok (-1.7tok)155 🟢55 🔴
Llm_calls1.55 (+0)1 🟢-
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens511.1tok (+4.23tok)43 🟢31 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens458.68tok (-26.01tok)122 🟢86 🔴
Completion_reasoning_tokens339.49tok (-26.2tok)102 🟢68 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens969.77tok (-21.79tok)123 🟢85 🔴
Estimated_cost0$ (0$)72 🟢51 🔴
Duration9.8s (-1.72s)156 🟢55 🔴
Llm_duration10.35s (-1.87s)156 🟢53 🔴
@delner
David Elner (delner)force-pushed
the
ci/release_shared_action
branch
3 times, most recently
from
429951e to
2841f76CompareJuly 16, 2026 03:55
@delner
David Elner (delner)
marked this pull request as ready for review
July 17, 2026 03:05
AbhiPrasad
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
View reviewed changes
Comment thread.github/workflows/publish.yaml
@AbhiPrasadAbhijeet Prasad (AbhiPrasad)Jul 17, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still need two different files for trusted publishing? Or is it that both python and js point to the same file?
@delnerDavid Elner (delner)Jul 23, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They point to the same file.
@delnerDavid Elner (delner)
mentioned this pull request
Jul 17, 2026
 Merged

@delner
David Elner (delner)
merged commit b0e1055
into
mainJul 24, 2026

30 checks passed

@github-actions
github-actionsBot
commented
Jul 24, 2026•

edited

Loading

Copy link


Copy Markdown

Braintrust eval report
Autoevals (main-1784905235)
ScoreAverageImprovementsRegressions
NumericDiff78.9% (0pp)13 🟢9 🔴
Time_to_first_token9.92tok (+0.12tok)113 🟢106 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens515.75tok (+4.65tok)20 🟢43 🔴
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens475.12tok (+16.44tok)97 🟢115 🔴
Completion_reasoning_tokens358.69tok (+19.2tok)78 🟢106 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens990.87tok (+21.1tok)96 🟢118 🔴
Estimated_cost0$ (+0$)56 🟢82 🔴
Duration9.92s (+0.12s)113 🟢106 🔴
Llm_duration10.84s (+0.49s)97 🟢122 🔴
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@AbhiPrasadAbhijeet Prasad (AbhiPrasad)AbhiPrasad approved these changes

Assignees

@delnerDavid Elner (delner)

Labels

enhancementNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad