Skip to content

fix(actionagent): register observed agents when a trace is ingested - #363

Merged
TonsOfFun merged 1 commit into
mainfrom
fix/register-agents-from-ingested-traces
Aug 15, 2026
Merged

fix(actionagent): register observed agents when a trace is ingested#363
TonsOfFun merged 1 commit into
mainfrom
fix/register-agents-from-ingested-traces

Conversation

@TonsOfFun

Copy link
Copy Markdown
Contributor

The bug

A self-hosted mount shows Agents: 0 while Traces and Interactions are full of that agent's runs.

AgentRegistrar has shipped with the engine since the split, is well written (identity is (owner, service_name, agent_class, action_name), caps observed records per owner, adopts the winner on a concurrent RecordNotUnique, never raises) — and is never called. It appears only in comments.

Its own docstring describes the exact failure:

Agents authored in the dashboard have records; agents running inside a customer's own app only ever reported traces, so the Agents list read 0 while Traces and Interactions were full of their runs.

Why it matters beyond the count

agent_id stays null on every ingested trace, so everything that scopes to an agent is empty: per-agent Traces, Interactions, Metrics, Evals, Versions, and the agent detail page itself. For a host app reporting telemetry — the whole self-hosted story — the Agents section is inert.

The fix

One line in create_from_payload, which is the single funnel every ingest path already goes through: the API controller, ProcessTelemetryTracesJob, AgentExecutionService, and the local_storage store.

create!(attrs).tap{ |record| AgentRegistrar.call(record)}

Note on the platform app

activeagents.ai carries this as a workaround in its own TelemetryTrace subclass, which is why the hosted dashboard lists agents and a self-hosted mount does not. That override can be dropped once this ships.

Verification

Against a self-hosted mount with local_storage: true, starting from an empty telemetry DB:

  • Ran CourseBuilderChatAgent (2 turns) and QuizBlueprintAgent (1) from a host app
  • Both registered themselves with correct agent_class_name / action_name, status observed
  • Every trace came back with agent_id set
  • Agent pages, per-agent Traces and per-agent Interactions all populated
  • A tool-calling turn then surfaced in the Tools view (regenerate_quiz, calls=1), which also depends on this attribution

Distinct actions on one class register separately, as designed — DocumentEnrichmentAgent.extract_page_summaries_batch and .extract_document_sections are different agents.

Tests

Three added to actionagent/test/telemetry_trace_test.rb: registration on ingest with attribution, reuse across repeated ingests, and that a registration failure does not fail ingest. The last stubs AgentRegistrar.new rather than .call, so the real rescue runs instead of the stub.

⚠️Not run locally — the dummy app's encrypted credentials fail to decrypt in my checkout (ActiveSupport::MessageEncryptor::InvalidMessage, no master key), which breaks existing tests on a clean tree too. Relying on CI here.

Risk

Low, and bounded by the registrar's own guarantees: it never raises, so a registration failure cannot cost a host app its telemetry; MAX_OBSERVED_PER_OWNER bounds runaway creation from a misconfigured reporter; and it returns early when agent_id is already set, so re-ingest is idempotent.

🤖 Generated with Claude Code

AgentRegistrar has existed since the engine shipped, is documented as the
thing that gives telemetry-only agents an identity, and is never called —
only referenced from comments. The result is the exact failure its own
docstring describes: an agent running inside a host app never authors a
dashboard record, so the Agents list reads 0 while Traces and Interactions
are full of that agent's runs.
Everything downstream of that identity is empty too. agent_id stays null on
every trace, so per-agent traces, interactions, metrics, evaluations and
versions have nothing to scope to, and the agent pages do not exist.
create_from_payload is the single funnel every ingest path goes through —
the API controller, ProcessTelemetryTracesJob, AgentExecutionService, and
the local_storage store — so registering there covers all of them.
The registrar was already written for this: identity is
(owner, service_name, agent_class, action_name), it caps observed records
per owner, it adopts the winner on a concurrent RecordNotUnique, and it
never raises. A regression test covers that last property by stubbing the
instance rather than the class method, so the real rescue runs.
Worth noting the platform app carries this as a workaround in its own
TelemetryTrace subclass, which is why the hosted dashboard lists agents and
a self-hosted mount does not. That override can be dropped once this ships.
Verified against a self-hosted mount: two agents registered themselves from
live traces with correct class/action, and their agent pages, per-agent
traces and interactions all populated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@TonsOfFun
TonsOfFun marked this pull request as ready for review August 14, 2026 23:49
@TonsOfFun
TonsOfFun merged commit 8ffe19d into mainAug 15, 2026
6 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@TonsOfFun