feat: add archive command for batch evaluations and recommendations

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.

[padmak30]

Cherrypick 7586092

• feat: add archive command for batch evaluations and recommendations. Introduces a new top-level archive command with two subcommands: archive batch-evaluation and archive recommendation. Each calls the corresponding service delete API and removes the matching local .cli/ history file.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	43.58%	9957 / 22847
🔵	Statements	42.86%	10578 / 24678
🔵	Functions	40.55%	1680 / 4143
🔵	Branches	40.26%	6426 / 15960

Generated in workflow #2398 for commit 2365b65 by the Vitest Coverage Report Action

[agentcore-cli-automation]

IAM policy changes go far beyond archive/delete and include a broken placeholder

docs/policies/iam-policy-user.json grows by 163 lines in this PR. Only the final BatchEvalAndRecommendation statement (with DeleteBatchEvaluation / DeleteRecommendation) is actually needed to support the new archive command. The rest adds a sweeping set of unrelated permissions to this preview branch:

• AgentCoreResourceManagement (runtime, memory, evaluator, gateway, workload identity CRUD)
• CloudFormationFull — cloudformation:* on *
• SsmParameterLookup, CloudFormationTemplateVerification
• ImportTestIam, ImportTestPassRole, ImportTestS3 — appear to be bugbash-only
• SecretsManager, CustomJwtCognitoSetup
• HarnessManagement, HarnessPassRole
• ConfigBundleManagement
• HttpGatewayIamRoleManagement

Two concrete problems:

1. Unreplaced placeholder. The ImportTestIam and ImportTestPassRole statements use "arn:aws:iam::ACCOUNT_ID:role/bugbash-agentcore-role" with a literal ACCOUNT_ID string. If a user applies this JSON directly, IAM will reject it as an invalid ARN. The other two policy files (iam-policy-boundary.json, iam-policy-cfn-execution.json) don't use this convention, so there's no documented substitution step either.
2. Scope creep on a cherry-pick. The PR description says this is a cherry-pick for the archive command. The preview branch's iam-policy-user.json is currently 140 lines; this would balloon it to 303 lines with statements that have nothing to do with archive. Several of these (Harness, ConfigBundle, CloudFormation:*) look like they belong in separate, reviewed PRs.

Options:

• (preferred) Drop every new statement except BatchEvalAndRecommendation from this PR, and open separate PRs for the other permission groups that actually need to land on preview.
• Keep only the statements that reflect features already present on the preview branch, remove the ImportTest* bugbash entries entirely, and replace ACCOUNT_ID with * (or document a substitution step).

[agentcore-cli-automation]

requireProject() runs outside the try/catch and breaks the --json contract

In src/cli/commands/archive/command.tsx, executeArchive calls requireProject() on line 21, before the try block:

asyncfunctionexecuteArchive<Textends{status: string}>(cliOptions: {id: string;region?: string;json?: boolean},config: { ... }): Promise<void>{requireProject();// <-- renders Ink + process.exit(1) on failuretry{constregion=awaitgetRegion(cliOptions.region);
...

requireProject() (in src/cli/tui/guards/project.tsx) renders an Ink <FatalError/> and calls process.exit(1) when there's no project or when the cwd isn't the project root. It has no awareness of --json.

So if a user runs agentcore archive batch-evaluation --id foo --json from outside a project (or from a subdirectory), they get Ink-formatted TTY output on stdout instead of the { "success": false, "error": "..." } envelope every other branch of this command produces. Scripted consumers relying on --json will fail to parse it. The e2e tests don't catch this because they run from the project root.

Options:

• Move the project check inside the try block, handle cliOptions.json explicitly, and emit a JSON error before exiting — mirroring the pattern already used in src/cli/commands/pause/command.tsx for the online-eval subcommand.
• Or, since the service-side delete doesn't actually require a local project (only the local .cli/ cleanup does), drop requireProject() entirely and let deleteLocalBatchEvalRun / deleteLocalRecommendationRun throw when there's no project — that error is already caught and surfaced as localDeleteWarning, so the service delete still succeeds and the user still gets a clean JSON response.

[padmak30]

For the 2 comments from Harness review

1. The additional IAM changes were added to fix a merge conflict, since both branches were out of sync for this file. The existing policy on main has ACCOUNT_ID, so keeping it for now. This policy is not used anywhere, only used for docs.
2. All other existing commands (recommendations history, evals history) call requireProject() outside the try/catch — same pattern as archive. None of them handle the --json contract on a project-not-found failure; they all fall through to requireProject's Ink render + process.exit(1). So keeping the behavior consistent with other patterns and main.

[agentcore-cli-automation]

Reviewed the diff. The two substantive issues I would have raised — the unrelated IAM policy additions / unreplaced ACCOUNT_ID placeholder in docs/policies/iam-policy-user.json, and requireProject() running outside the try/catch in executeArchive breaking the --json contract — are already flagged in the existing review comments, and the author has responded with their rationale. No additional serious issues to call out. LGTM.