Uh oh!
There was an error while loading. Please reload this page.
feat: instrument telemetry for dev command - #1223
Conversation
Package TarballHow to installnpm install https://github.com/aws/agentcore-cli/releases/download/pr-1223-tarball/aws-agentcore-0.13.1.tgz |
94547e7 to
fa268adCompare
agentcore-cli-automation
left a comment
There was a problem hiding this comment.
Telemetry instrumentation looks good overall, but a few concerns about the SIGINT/exit handling for the long-running server modes that I think need to be resolved before merging. The invoke and exec paths look clean.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
fa268ad to
b9c1ec1Compareb9c1ec1 to
a3eadeeComparea3eadee to
160dbbcCompare160dbbc to
5c1043cCompare
agentcore-cli-automation
left a comment
There was a problem hiding this comment.
All four serious issues raised in earlier review comments appear to be addressed in the latest commit (53c5d65):
--logsSIGINT cleanup race —resolve()is no longer called from the SIGINT handler at lines 407–411; the promise now only resolves viaonExit, so SIGTERM (with the 2s SIGKILL fallback inDevServer.kill) has time to drive the child to exit beforeprocess.exit(0)runs.- Browser-mode SIGINT race — the duplicate
process.once('SIGINT', ...) { resolve() }was removed (lines 472–493). The code now just awaitsrunBrowserModeand includes an inline comment explicitly calling out the limitation that normal shutdown telemetry requires a follow-up refactor ofrunWebUI. Reasonable interim state. - TUI
process.exit(0)fromonBack— explicitcollector?.stop(); process.exit(0)was added after the wrapper at lines 463–464, giving the--no-browserpath a deterministic exit point. --logsfailure path silently exits — replaced withif (!devResult.success) throw devResult.error; process.exit(0);at lines 416–417, matching the pattern used by the other paths and surfacing the error via the outercatch.
The integ test exercises both success and failure telemetry through a real audit dir (no mocking of telemetry internals), which is good. Schema additions (exec action, UiMode, agui protocol, ui_mode field) and unit tests look correct.
LGTM.
95de36f to
a74d5b1Comparea74d5b1 to
56f64f6Compare56f64f6 to
3905aeeCompare…metry eagerly Error classification: - ConnectionError for connection-refused (new class in lib/errors/types.ts) - ValidationError for invalid user input (missing --tool, bad JSON, unknown command) - ResourceNotFoundError for missing container runtime Browser mode telemetry: - Emit telemetry eagerly via TelemetryClientAccessor before the blocking runBrowserMode call (which never returns). Do not copy this pattern — prefer withCommandRunTelemetry for commands that return.
3905aee to
88b2a30CompareUh oh!
There was an error while loading. Please reload this page.
Description
Add telemetry instrumentation to the
devcommand for all execution paths (invoke, exec, server modes).Schema changes:
Actionenum:'exec'UiModeenum:'browser' | 'terminal''agui'toProtocolenumui_modefield toDevAttrsAttributes emitted: action, ui_mode, has_stream, protocol, invoke_count
Note: because the dev server runs indefinitely, we emit telemetry eagerly before the server starts. The alternative is to refactor how runWebUI works to return a result that allows us to determine if the error was a cancellation (ctrl + c) which would count as a success, or a crash, which would count as a failure. However, this is a large refactor and is therefore left out of scope.
The effect is that dev success metrics with browser correspond to "was the user able to launch the browser" rather than, "did the browser launch successfully".
Related Issue
Closes #
Documentation PR
N/A
Type of Change
Testing
How have you tested the change?
npm run test:unitandnpm run test:integnpm run typechecknpm run lintsrc/assets/, I rannpm run test:update-snapshotsand committed the updated snapshotsAdditional testing:
Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.