feat(engine): add evaluation runtime tuning by abhinav-galileo · Pull Request #239 · agentcontrol/agent-control

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo · 2026-06-15T15:04:36Z

Summary:

Exposes evaluation concurrency as a configurable engine setting.
Adds Luna HTTP connection and client-pool tuning.
Preserves Luna HTTP error metadata for evaluator failures.

Validation:

git diff --check passed for the split diff.
make engine-test did not reach tests because uv timed out fetching greenlet from the package index.

codecov · 2026-06-15T16:07:32Z

Codecov Report

❌ Patch coverage is 97.34513% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
engine/src/agent_control_engine/core.py	85.71%	2 Missing ⚠️
.../agent_control_evaluator_galileo/luna/evaluator.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

namrataghadi-galileo · 2026-06-17T05:00:00Z

[client.py:87 / core.py:31 / server] Three divergent env-int parsers across packages

_load_int_env (client.py), _env_positive_int (core.py), and presumably a third in the server package all parse integers from env vars but differ on: empty-string handling, exception type (ValueError vs RuntimeError), and positivity validation. Any fix to one is invisible to the others.

abhinav-galileo · 2026-06-17T08:12:26Z

For the env parser consistency note: I fixed the immediate Luna parser inconsistency for empty values in 35a12c1. I did not extract a shared env parser across packages in this PR to avoid broadening the runtime-tuning change.

## [3.1.0](ts-sdk-v3.0.0...ts-sdk-v3.1.0) (2026-06-17) ### Features * **engine:** add evaluation runtime tuning ([#239](#239)) ([2f1e5fe](2f1e5fe)) * **server:** bound database connection pool ([#236](#236)) ([80aea8f](80aea8f)) ### Bug Fixes * **evaluators:** configure Luna scorer API URL ([#237](#237)) ([fc516f0](fc516f0)) * **server:** use plain psycopg with OS libpq ([#241](#241)) ([45a33a5](45a33a5))

galileo-automation · 2026-06-17T15:58:03Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

abhinav-galileo mentioned this pull request Jun 15, 2026
feat(server): add evaluation timing metrics #238
Closed

abhinav-galileo changed the title ~~feat: add evaluation runtime tuning~~feat(engine): add evaluation runtime tuningJun 15, 2026

abhinav-galileo marked this pull request as ready for review June 15, 2026 15:35

abhinav-galileo force-pushed the abhi/agent-control-runtime-tuning branch from a3606fa to 9b234e4Compare June 15, 2026 16:02

abhinav-galileo requested a review from namrataghadi-galileo June 15, 2026 17:47

namrataghadi-galileo reviewed Jun 17, 2026
View reviewed changes

Comment threadevaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py

namrataghadi-galileo reviewed Jun 17, 2026
View reviewed changes

Comment threadevaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py

namrataghadi-galileo reviewed Jun 17, 2026
View reviewed changes

Comment threadevaluators/contrib/galileo/src/agent_control_evaluator_galileo/luna/client.py Outdated

abhinav-galileo force-pushed the abhi/luna-internal-api-url branch from 44185bb to 1a18fccCompare June 17, 2026 12:12

Base automatically changed from abhi/luna-internal-api-url to mainJune 17, 2026 12:31

abhinav-galileo added 6 commits June 17, 2026 18:06

feat(evaluator-galileo): add Luna HTTP connection tuning
f089287

feat(engine): expose evaluation concurrency setting
98d79d0

feat: add Luna HTTP client pool setting
44b95fd

fix: expose luna http error metadata
a3847ea

docs(engine): clarify concurrency env fallback
83873bb

fix(evaluators): synchronize Luna client pooling
664a78f

abhinav-galileo force-pushed the abhi/agent-control-runtime-tuning branch from 35a12c1 to 664a78fCompare June 17, 2026 12:37

abhinav-galileo requested a review from namrataghadi-galileo June 17, 2026 12:42

namrataghadi-galileo approved these changes Jun 17, 2026
View reviewed changes

abhinav-galileo merged commit 2f1e5fe into mainJun 17, 2026
6 checks passed

abhinav-galileo deleted the abhi/agent-control-runtime-tuning branch June 17, 2026 15:35

galileo-automation added the released label Jun 17, 2026