docs: fix 30 documentation inaccuracies found by source code audit by aidandaly24 · Pull Request #697 · aws/agentcore-cli

aidandaly24 · 2026-03-27T04:40:56Z

Description

Comprehensive audit of all .md documentation files against actual source code (Zod schemas, CLI commands, CDK constructs). Fixes 70+ inaccuracies across 14 files where docs had drifted from the implementation, including alignment with several upstream refactors and documentation of new features merged after the initial audit.

Upstream refactors accounted for

PR	Change	Doc Impact
#709	Remove `type` fields from resource schemas, rename credential `type` → `authorizerType`	Removed stale `type` fields from all JSON examples; renamed credential discriminator
#705	`add identity` → `add credential`, `--force` → `-y/--yes`, `--plan` → `--dry-run`	Updated all command references and flag tables
#706	`agents` → `runtimes` array, `--agent` → `--runtime` flags	Updated all flag tables, examples, and JSON config across 7 doc files
#707	Dev command `--invoke` → positional `[prompt]` arg	Already reflected in docs
#713	Revert CUSTOM strategy and semanticOverride	No doc changes needed (CUSTOM was never documented)

New features documented (merged after initial audit)

PR	Feature	Doc Files
#763, #780	Import subcommands (`import runtime`, `import memory`, `import evaluator`, `import online-eval`)	`commands.md`
#750	Exec mode for invoke and dev (`--exec`, `--timeout`)	`commands.md`, `local-development.md`
#739	Code-based evaluator support (`--type`, `--lambda-arn`, `--timeout`)	`commands.md`, `evals.md`
#732	Ground truth reference inputs (`--assertion`, `--expected-trajectory`, `--expected-response`)	`commands.md`, `evals.md`
#531, #534	Memory record streaming (`--data-stream-arn`, `--stream-content-level`, `--delivery-type`)	`commands.md`, `memory.md`
#782	`--skip-install` flag on `create`	`commands.md`
#774	`--identity-name` option on `fetch access`	`commands.md`

Key fixes by category

Schema/config drift (configuration.md, gateway.md, container-builds.md):

Added missing EPISODIC memory strategy and reflectionNamespaces field
Added lifecycleConfiguration (idleRuntimeSessionTimeout, maxLifetime) to agent spec
Added description, dockerfile, executionRoleArn to agent spec table
Fixed CUSTOM_JWT fields: allowedAudience/allowedClients/allowedScopes/customClaims are conditionally required
Fixed OAuth credential discoveryUrl: optional, not required (for imported providers)
Fixed memory strategies: defaults to empty array, not required
Added missing tags field to memory, credential, gateway, and online eval config tables
Added missing customClaims field to CUSTOM_JWT config
Fixed MCP Server outbound auth: removed api-key (only oauth/none valid per schema)
Added enableSemanticSearch, exceptionLevel, policyEngineConfiguration gateway fields
Fixed gateway target type count: six types (including lambda), not five
Removed stale "type": "AgentCoreRuntime" / "AgentCoreMemory" / "CustomEvaluator" / "OnlineEvaluationConfig" from all JSON examples
Fixed mcp.json reference → agentcore.json in local-development.md

Flag renames (commands.md, evals.md, local-development.md):

--agent → --runtime, -a → -r across all commands (status, dev, invoke, logs, traces, run eval, evals history, package, add online-eval, logs evals)
--agent-arn → --runtime-arn, --agent-runtime-id → --runtime-id
--plan → --dry-run, --force → -y/--yes
add identity → add credential, remove identity → remove credential
--agent-client-id → --client-id, --agent-client-secret → --client-secret (gateway CUSTOM_JWT)
--exception-level valid values: NONE/DEBUG (not ALL)
--lines → --limit in logs evals
Note: --agent-id and --agent-alias-id are unchanged (Bedrock Agent import flags)

Missing flags added (commands.md, evals.md):

add agent: --authorizer-type, --discovery-url, --allowed-audience, --allowed-clients, --allowed-scopes, --custom-claims, --client-id, --client-secret, --idle-timeout, --max-lifetime
create: --idle-timeout, --max-lifetime, --skip-install
add gateway: --runtimes, --custom-claims, --policy-engine, --policy-engine-mode
dev and invoke: -H, --header, --exec
invoke: --bearer-token, --exec, --timeout
add evaluator: --type, --lambda-arn, --timeout (code-based evaluator support)
run eval: --assertion, --expected-trajectory, --expected-response, --endpoint
add memory: --data-stream-arn, --stream-content-level, --delivery-type, --stream-delivery-resources
fetch access: --identity-name
gateway.md: --oauth-scopes in OAuth example

New sections added:

commands.md: Full import command section with 4 subcommands (runtime, memory, evaluator, online-eval)
evals.md: Evaluator Types section (LLM-as-a-Judge vs code-based), Ground Truth Reference Inputs subsection, Builtin Evaluator table (all 10 IDs with levels)
evals.md: Fixed eval results storage path (eval-runs → eval-results)
memory.md: Memory Record Streaming section with CLI flags and JSON config example
local-development.md: Exec Mode section covering dev exec, invoke exec, and TUI shortcuts

Stale values (commands.md, evals.md, frameworks.md):

Fixed model ID claude-3-5-sonnet-20250929 → claude-3-5-sonnet-20250514
Added policy-engine/policy to status --type enum
Removed ghost --tool-filter-description flag (not in schema or CLI)
Fixed evaluator level values: added TOOL_CALL, removed SPAN

Structural/naming (AGENTS.md, README.md, assets):

Removed AutoGen from supported frameworks list (5 frameworks, not 6)
Fixed "agents" → "runtimes" in gateway JSON example
Renamed identity → credential in all command references
Fixed file paths and directory references
Fixed regex constraints and enum values to match Zod schemas

Files modified

File	Changes
`docs/configuration.md`	Schema field tables, JSON examples, type removals, credential rename, tags fields, required/optional fixes
`docs/commands.md`	All flag tables and examples, import subcommands, exec mode, streaming, evaluator types, ground truth
`docs/evals.md`	Flag tables, examples, evaluator types section, ground truth inputs, builtin evaluator table, eval-results path fix
`docs/gateway.md`	identity → credential, agents → runtimes, oauth-scopes, target type count
`docs/local-development.md`	--runtime flag in dev examples, exec mode section, mcp.json → agentcore.json
`docs/memory.md`	Removed stale types, added memory record streaming section
`docs/container-builds.md`	Removed stale `"type": "AgentCoreRuntime"` from JSON example
`docs/frameworks.md`	Model ID fix, framework list correction
`AGENTS.md`	identity → credential in primitives
`README.md`	identity → credential
`src/assets/README.md`	identity → credential in vended template
`src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap`	Updated snapshot

Known source code issue (not fixed here)

src/cli/primitives/GatewayPrimitive.ts has incorrect CLI help text: --exception-level says ALL (should be DEBUG) and --authorizer-type omits AWS_IAM. This is a source code bug that should be addressed in a separate PR.

Related Issue

N/A — discovered via systematic source code audit

Documentation PR

N/A — this PR is the documentation fix

Type of Change

Testing

I ran npm run test:unit and npm run test:integ
I ran npm run typecheck
I ran npm run lint
If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

All changes are documentation-only (.md files + snapshot update). Each fix was verified against the corresponding Zod schema definition or CLI source code (src/cli/commands/*/command.tsx, src/cli/primitives/*.ts).

Note: 19 pre-existing test failures exist on upstream/main from the schema refactor (#709) — not introduced by this PR. The 2 e2e failures are expected on fork PRs (no AWS credentials).

Checklist

I have read the CONTRIBUTING document
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

github-actions · 2026-03-27T17:37:45Z

Package Tarball

aws-agentcore-0.7.1.tgz

How to install

npm install https://github.com/aws/agentcore-cli/releases/download/pr-697-tarball/aws-agentcore-0.7.1.tgz

- configuration.md: fix networkMode PRIVATE→VPC, remove stale mcp.json section, add 7 missing AgentEnvSpec fields, add Node.js runtimes, fix gateway name max 63→100, list all 6 targetType values, fix memory strategies wording, fix deployed-state.json path, add missing project fields, fix CUSTOM_JWT field requirements, add customClaims field - README.md: add CrewAI framework, update all 4 default model IDs, add missing command sections - frameworks.md: add CrewAI with model providers, add protocol compatibility matrix (HTTP/MCP/A2A) - commands.md: fix agent name max 64→48 chars, add fetch access section, add policy-engine/policy to status --type, remove ghost --tool-filter-description flag, fix model ID date - gateway.md: add enableSemanticSearch/exceptionLevel config options, fix mcp.json→agentcore.json, remove api-key from MCP Server auth - transaction_search.md: fix traces command syntax - container-builds.md: remove nonexistent --progress flag - evals.md: fix model date 20250929→20250514 - local-development.md: fix mcp.json→agentcore.json - AGENTS.md: remove AutoGen, remove stale mcp.json reference - src/assets/README.md: add CrewAI to template list - src/assets/agents/AGENTS.md: remove AutoGen Constraint: All fixes verified against Zod schemas and CLI runtime Constraint: CLI --help text bugs left for separate agent Confidence: high Scope-risk: narrow

- Remove `type` fields from agent, memory, evaluator, online-eval-config examples and field tables (removed in aws#709) - Remove `modelProvider` from agent spec (removed in aws#709) - Rename credential `type` to `authorizerType` (renamed in aws#709) - Rename `add identity` to `add credential` everywhere (aws#705) - Replace `--force` with `-y/--yes` on remove commands (aws#705) - Replace `--plan` with `--dry-run` on deploy command (aws#705) - Update snapshot for src/assets/README.md change

Update `"agents": [...]` to `"runtimes": [...]` in the gateway configuration example to match the schema rename in aws#706.

Update all documentation to reflect upstream renames: - --agent → --runtime, -a → -r across all commands - --agent-arn → --runtime-arn, --agent-runtime-id → --runtime-id - Remove stale "type" fields from JSON examples in evals.md, memory.md, and container-builds.md Constraint: --agent-id, --agent-alias-id, --agent-client-id, --agent-client-secret are unchanged (different semantics) Constraint: --type agent filter value in status/fetch stays as-is Confidence: high Scope-risk: narrow

- Add -H, --header flag to dev and invoke commands - Add --bearer-token flag to invoke command - Fix --lines → --limit in evals.md logs evals section Confidence: high Scope-risk: narrow

- Rename --agent-client-id → --client-id, --agent-client-secret → --client-secret in gateway docs (naming unification aws#705) - Add missing flags to add agent: --authorizer-type, --discovery-url, --allowed-audience, --allowed-clients, --allowed-scopes, --custom-claims, --client-id, --client-secret, --idle-timeout, --max-lifetime - Add missing flags to create: --idle-timeout, --max-lifetime - Add missing flags to add gateway: --runtimes, --custom-claims, --policy-engine, --policy-engine-mode - Fix --exception-level values: NONE/ALL (was undocumented) Confidence: high Scope-risk: narrow

The CLI help text says ALL but the schema (GatewayExceptionLevelSchema) defines the valid values as NONE and DEBUG. Aligns commands.md with gateway.md and the Zod schema.

- Remove stale "type": "AgentCoreMemory" from memory streaming example - Add memory streaming flags: --delivery-type, --data-stream-arn, --stream-content-level, --stream-delivery-resources - Add --exec to dev command flag table (Container agents) - Add --skip-install to create command flag table - Add --identity-name to fetch access flag table

Hweinstock · 2026-04-09T16:46:25Z

Looks like lint is failing, otherwise looks good:

Run npm run format:check
> @aws/agentcore@0.7.1 format:check
> prettier --check .
Checking formatting...
[warn] docs/commands.md
[warn] Code style issues found in the above file. Run Prettier with --write to fix.

jariy17

lgtm and fix the lint

aidandaly24 · 2026-03-27T04:40:56Z