Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Use shared release actions for JS package release by delner · Pull Request #202 · braintrustdata/autoevals · GitHub
Skip to content

Use shared release actions for JS package release - #202

Closed
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action
Closed

Use shared release actions for JS package release#202
David Elner (delner) wants to merge 1 commit into
mainfrom
ci/release_js_with_shared_action

Conversation

@delner

@delnerDavid Elner (delner) commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Moves the autoevals npm publish onto the canonical, centrally-maintained release actions in braintrustdata/sdk-actions (pinned by SHA), and switches releases from branch-based to explicit-SHA dispatch so js and py publish the same reviewed commit.

Changes

  • publish-js.yaml — rewritten to wrap the shared actions: compute-metadata → validate → prepare → notify-pending → publish. autoevals keeps only its glue (version-sync, version/channel computation, the prerelease package.json patch); the shared jobs are just the pinned action + inputs. Adds:
    • an approval gate (npm-publish / npm-publish-dry-run environments),
    • Slack pending + completion notifications,
    • a dry_run input (build/pack, no publish).
  • publish.yaml (dispatcher) + publish-py.yamlbranch input → required sha; the same commit is passed to both languages. Python's publish mechanics are otherwise unchanged (no shared-action adoption yet).
  • docs/PUBLISHING.md: updated for the shared actions, the gate, dry_run, Slack, the npm trusted-publisher environment requirement, and branch → sha.

Unchanged: version-sync.yaml, check_version_sync.py, OIDC trusted publishing + provenance, tag scheme (js-/py-), pnpm/node versions, PyPI publishing.

Required setup before the first real publish

  • GitHub Environments: npm-publish (required reviewers, main-only) and npm-publish-dry-run.
  • Update the npm trusted publisher to include the npm-publish environment: the gated job's OIDC token carries an environment claim. dry_run does not exercise this..
  • Slack: SLACK_SDK_RELEASE_CHANNEL var.

Note

JS gains the approval gate + Slack; Python does not yet (follow-up once shared Python release actions exist).

@github-actions

github-actionsBot commented Jun 23, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1782772285)

ScoreAverageImprovementsRegressions
NumericDiff78.5% (+1pp)11 🟢4 🔴
Time_to_first_token10.4tok (-0.27tok)132 🟢87 🔴
Llm_calls1.55 (+0)--
Tool_calls0 (+0)--
Errors0 (+0)--
Llm_errors0 (+0)--
Tool_errors0 (+0)--
Prompt_tokens528.42tok (+0tok)--
Prompt_cached_tokens0tok (+0tok)--
Prompt_cache_creation_tokens0tok (+0tok)--
Prompt_cache_creation_5m_tokens0tok (+0tok)--
Prompt_cache_creation_1h_tokens0tok (+0tok)--
Completion_tokens466.42tok (+3.81tok)95 🟢107 🔴
Completion_reasoning_tokens354.33tok (+3.78tok)79 🟢90 🔴
Completion_accepted_prediction_tokens0tok (+0tok)--
Completion_rejected_prediction_tokens0tok (+0tok)--
Completion_audio_tokens0tok (+0tok)--
Total_tokens994.85tok (+3.81tok)95 🟢107 🔴
Estimated_cost0$ (+0$)78 🟢94 🔴
Duration10.4s (-0.27s)132 🟢87 🔴
Llm_duration11.31s (-0.37s)135 🟢84 🔴

@delner

Copy link
Copy Markdown
CollaboratorAuthor

This is based on the HEAD of braintrustdata/sdk-actions#55 currently: we need to merge that and bump SHAs before we merge this one.

Comment on lines +172 to +173
slack_token: ${{ secrets.SLACK_BOT_TOKEN }}
slack_channel: ${{ vars.SLACK_SDK_RELEASE_CHANNEL }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's kind of annoying that you need to set up a slack token per repo.

Maybe we can make this action execute a trigger on the sdk-actions repo and have it trigger the slack action there? Might be too involved/complicated.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set it up per repo if it's org level.

Comment thread.github/workflows/publish-js.yaml Outdated
@delner
David Elner (delner)force-pushed the ci/release_js_with_shared_action branch from a8cf9bc to b59171aCompareJune 24, 2026 14:41
@delner

Copy link
Copy Markdown
CollaboratorAuthor

Closed in favor of #204

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@delner@AbhiPrasad