feat: dataset management — full feature with tests by jariy17 · Pull Request #1347 · aws/agentcore-cli

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 · 2026-05-21T16:43:56Z

Summary

Implements complete dataset management lifecycle for AgentCore CLI:

Schema & Primitives: DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types, name validation, kmsKeyArn support
CLI Commands: add dataset, remove dataset, dataset download, dataset publish-version, dataset remove-version
Operations: Incremental push engine (diff by exampleId, batched API calls with retry), post-deploy sync (hash-based change detection), dataset-driven evaluation (load scenarios, invoke agent, collect spans)
TUI: Add dataset wizard, dataset hub (download/publish/remove-version), dataset picker in batch eval flow
Bug Fixes: Deploy preflight now recognizes datasets as deployable resources; removed stale x-amz-expected-bucket-owner header workaround

Test plan

68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight)
17 integration tests (undeployed commands, validation edge cases, scaffolding verification, file preservation)
19 E2E tests (full lifecycle, 1000-example large batch, dataset-driven eval with Builtin.Faithfulness)
All E2E tests verified in ap-southeast-2 against account 619071331382

Implements complete dataset management lifecycle for AgentCore: Schema & Primitives: - DatasetPrimitive (add/remove/preview) with PREDEFINED_V1 and SIMULATED_V1 schema types - Dataset schema validation (name, schemaType, config.managed.location, kmsKeyArn) - CDK AgentCoreDataset construct (AWS::BedrockAgentCore::Dataset) CLI Commands: - `agentcore add dataset` / `agentcore remove dataset` - `agentcore dataset download` — pull from service to local file - `agentcore dataset publish-version` — publish DRAFT as immutable version - `agentcore dataset remove-version` — delete published version Operations: - Incremental push engine (diff by exampleId, batched API calls with retry) - Post-deploy sync (hash-based change detection, automatic on deploy) - Dataset-driven evaluation (load scenarios, invoke agent, collect spans) - resolve-dataset, wait, status, pull utilities TUI: - Add dataset wizard (name → schema type → description → confirm) - Dataset hub (download/publish/remove-version management) - Dataset picker in batch eval flow with version selection Bug Fixes: - Deploy preflight now recognizes datasets as deployable resources - Removed stale x-amz-expected-bucket-owner header workaround Tests: - 68 unit tests (DatasetPrimitive, AWS client, push engine, pull, publish, resolve, wait, post-deploy sync, session provider, preflight) - 17 integration tests (undeployed commands, validation, scaffolding) - 19 E2E tests (lifecycle, large batch 1000 examples, eval integration)

agentcore-devx-automation · 2026-05-21T16:49:10Z

Claude Security Review: no high-confidence findings. (run)

- Replace non-existent `cliCommandRun` import with `runCliCommand` - Replace non-existent `RemovalResult` type with `Result` from lib - Fix error type mismatches (string vs Error object) across dataset primitives, TUI flows, and batch evaluation - Add `result.targetName` guard in status command to prevent undefined index - Remove unused `DEFAULT_ENDPOINT_NAME` import in get-trace.ts - Fix prettier formatting in invoke/action.ts and eval/run-eval.ts - Update asset snapshots and test assertions to match Result<T,E> type

github-actions · 2026-05-21T18:14:24Z

Package Tarball

aws-agentcore-0.14.2.tgz

How to install

gh release download pr-1347-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.14.2.tgz

agentcore-devx-automation · 2026-05-21T18:16:51Z

Claude Security Review: the review run failed before completing. See the run for details.

github-actions · 2026-05-21T18:17:24Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	44.57%	9918 / 22249
🔵	Statements	43.82%	10535 / 24040
🔵	Functions	41.17%	1724 / 4187
🔵	Branches	40.89%	6346 / 15516

Generated in workflow #3229 for commit a24b351 by the Vitest Coverage Report Action

tejaskash

Nice work overall — clean layering, good test coverage. A few inline notes; only the post-deploy hash one feels like it should block.

- Recompute content hash after pushDataset to avoid stale hash causing redundant network calls on every deploy - Replace Promise.all with Promise.allSettled in span-collector so one failed session doesn't abort the entire eval run - Move getDataset to static import in publish.ts (no cycle to dodge) - Add AbortSignal.timeout(30s) to fetch in agentcore-datasets.ts so hung connections don't block indefinitely

agentcore-devx-automation · 2026-05-21T20:15:30Z

Claude Security Review: the review run failed before completing. See the run for details.

tejaskash

Verified the four blocking/sub-blocking comments are addressed:

post-deploy-datasets.ts: now re-reads the file after pushDataset and hashes the post-push content ✓
span-collector.ts: switched to Promise.allSettled so one failure doesn't abort the rest ✓
publish.ts: dynamic getDataset import promoted to top-level ✓
agentcore-datasets.ts: fetch() now uses AbortSignal.timeout(30_000) ✓

LGTM.

agentcore-devx-automation · 2026-05-22T02:03:57Z

Claude Security Review: no high-confidence findings. (run)

padmak30

Some of the back navigation is broken.

From run -> batch evals -> eval source (dataset & version) -> evaluators and navigating back with escape takes me back to run skipping the eval source step.
Back-navigation in AddDatasetScreen loses previously typed name.
DatasetFlow back-nav resets datasets to [], leaving the hub permanently stuck with "No items available" after pressing Esc from any sub-screen.

* fix(tui): back-navigation bugs in dataset and batch eval flows Three back-navigation issues reported on PR #1347: 1. BatchEvalWizard Escape skips source-picker — onExit passed the top-level exit handler instead of returning to the source-picker state. Now returns to source-picker on first-step back-nav. 2. AddDatasetScreen loses typed name — initialValue always called generateUniqueName() ignoring the stored name state. Now uses the previously submitted name when available. 3. DatasetFlow back-nav resets datasets to [] — all four sub-screen onExit/onCancel handlers hardcoded an empty array. Added loadedDatasets state that persists across flow transitions. * feat(dataset): add --kms-key-arn support to add dataset command and TUI Wire kmsKeyArn through the full add-dataset flow: - CLI: --kms-key-arn flag with ARN validation - TUI: optional step after Description (Enter to skip) - Primitive: writes kmsKeyArn to agentcore.json when provided - CDK construct already passes KmsKeyArn to CloudFormation The field is immutable after dataset creation (createOnly CFN property). * test(dataset): add kmsKeyArn unit and integration tests Cover the kmsKeyArn flow with: - Unit: DatasetPrimitive.add() persists kmsKeyArn when provided, omits when absent - Integ: CLI --kms-key-arn writes to agentcore.json, invalid ARN rejected, absent ARN omitted --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

jariy17 requested a review from a teamMay 21, 2026 16:43

github-actionsBot added the size/xl PR size: XL label May 21, 2026

jariy17 had a problem deploying to e2e-testing May 21, 2026 16:44 — with GitHub Actions Failure

agentcore-devx-automationBot added the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

github-actionsBot added the agentcore-harness-reviewing AgentCore Harness review in progress label May 21, 2026

github-code-qualityBot found potential problems May 21, 2026
View reviewed changes

Comment threadsrc/cli/commands/dataset/command.tsx Fixed
Comment threadsrc/cli/commands/dataset/command.tsx Fixed
Comment threadsrc/cli/commands/dataset/command.tsx Fixed
Comment threadsrc/cli/primitives/DatasetPrimitive.ts Fixed

agentcore-devx-automationBot removed the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

github-actionsBot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 21, 2026

github-actionsBot removed the size/xl PR size: XL label May 21, 2026

jariy17 had a problem deploying to e2e-testing May 21, 2026 18:13 — with GitHub Actions Failure

github-actionsBot added the size/xl PR size: XL label May 21, 2026

agentcore-devx-automationBot added the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

agentcore-devx-automationBot removed the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

tejaskash reviewed May 21, 2026
View reviewed changes

jariy17 had a problem deploying to e2e-testing May 21, 2026 20:13 — with GitHub Actions Failure

github-actionsBot added size/xl PR size: XL and removed size/xl PR size: XL labels May 21, 2026

agentcore-devx-automationBot added the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

agentcore-devx-automationBot removed the claude-security-reviewing Claude Code /security-review in progress label May 21, 2026

tejaskash approved these changes May 21, 2026
View reviewed changes

agentcore-devx-automationBot added the claude-security-reviewing Claude Code /security-review in progress label May 22, 2026

agentcore-devx-automationBot removed the claude-security-reviewing Claude Code /security-review in progress label May 22, 2026

padmak30 reviewed May 22, 2026
View reviewed changes

Comment threadsrc/schema/schemas/primitives/dataset.ts

jariy17 had a problem deploying to e2e-testing May 22, 2026 12:25 — with GitHub Actions Failure

jariy17 temporarily deployed to e2e-testing May 22, 2026 12:29 — with GitHub Actions Inactive

jariy17 temporarily deployed to e2e-testing May 22, 2026 12:34 — with GitHub Actions Inactive

jariy17 temporarily deployed to e2e-testing May 22, 2026 12:54 — with GitHub Actions Inactive

jariy17 merged commit 6043caa into mainMay 22, 2026
39 of 42 checks passed

jariy17 deleted the feature/dataset-management branch May 22, 2026 12:56

jariy17 mentioned this pull request May 26, 2026
fix(dataset): tui back-navigation bugs + kmsKeyArn support #1379
Merged
9 tasks