Skip to content

ci: install deepeval and autoevals for evaluation integ tests - #608

Merged
jariy17 merged 1 commit into
mainfrom
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged

ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom
fix/eval-integ-third-party-deps

Conversation

@jariy17

@jariy17jariy17 commented Jul 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Problem

Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):

ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.

Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.

Change

One line — add both packages to the evaluation group's extra-deps.

Testing

  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above

jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17 requested a review from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot added the size/xs PR size: XS label Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot added the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@agentcore-devx-automation

agentcore-devx-automationBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automationagentcore-devx-automationBot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@github-actionsgithub-actionsBot added size/xs PR size: XS and removed size/xs PR size: XS labels Jul 30, 2026
This was referenced Jul 30, 2026
Merged
Merged
@jariy17
jariy17 merged commit 5d23292 into mainJul 31, 2026
29 of 34 checks passed
@jariy17
jariy17 deleted the fix/eval-integ-third-party-deps branch July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees

No one assigned

Labels

size/xsPR size: XS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub
Skip to content
ci: install deepeval and autoevals for evaluation integ tests - #608
Merged
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged
ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom 
fix/eval-integ-third-party-deps
Conversation
@jariy17
@jariy17jariy17
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Problem
Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.
Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.
Change
One line — add both packages to the evaluation group's extra-deps.
Testing
  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above
jariy17
ci: install deepeval and autoevals for evaluation integ tests
b141570
tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.
Both packages are already declared as optional extras in pyproject.toml.
@jariy17
jariy17
requested a review
from a teamJuly 30, 2026 22:33
@github-actionsgithub-actionsBot
added
the 
size/xs
PR size: XS
label
Jul 30, 2026
@agentcore-devx-automationagentcore-devx-automationBot
added
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@agentcore-devx-automation
agentcore-devx-automationBot
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
Claude Security Review: no high-confidence findings. (run)
@agentcore-devx-automationagentcore-devx-automationBot
removed
the 
claude-security-reviewing
Claude Code /security-review in progress
label
Jul 30, 2026
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
Jul 30, 2026

This was referenced Jul 30, 2026
 Merged

 Merged

@jariy17
jariy17
merged commit 5d23292
into
mainJul 31, 2026

29 of 34 checks passed

@jariy17
jariy17
deleted the

fix/eval-integ-third-party-deps

branch
July 31, 2026 14:16
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@tejaskashtejaskashtejaskash approved these changes

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jariy17@tejaskash