Uh oh!
There was an error while loading. Please reload this page.
test(integ): expand harness config-shape coverage - #1608
Conversation
Add an integration suite covering the harness config surface the existing tests miss — all local (no AWS), exercising CLI flag → action → validate → spec-write wiring that schema unit tests don't reach: - model: lite_llm (+api-base/additional-params), bedrock api-format, plus api-format provider-gating rejections (open_ai/gemini) - tools (`add tool`): remote_mcp, code_interpreter, gateway (awsIam), inline_function, plus required-config rejections - skills (`add skill`): path / s3 / git (+auth credential) - memory modes (gated): managed + strategies, existing-by-name + tuning, plus by-arn / no-ref / gated-off rejections - truncation: summarization - VPC network-mode validation (no deploy) - CUSTOM_JWT inbound authorizer + missing-config rejection Cases share one project (single `create` spawn) and use it.each tables for rejection groups to keep the suite fast. Pure schema refinements already covered by harness.test.ts unit tests are intentionally not duplicated. All 35 tests in the file pass.
Claude Security Review: no high-confidence findings. (run) |
Package TarballHow to installgh release download pr-1608-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.20.2.tgz |
agentcore-cli-automation
left a comment
There was a problem hiding this comment.
Tests are well-structured and pragmatically scoped — sharing one project, real CLI spawns, and it.each rejection tables strikes a good balance. One concern below worth addressing before merge.
Uh oh!
There was an error while loading. Please reload this page.
Coverage Report
|
…FEATURES
The 'rejects --memory-mode when gated off' case relied on the gate being
unset in the environment. cleanSpawnEnv inherits the host process.env and
does not strip ENABLE_GATED_FEATURES, so a shell/CI with it exported as '1'
would flip the case: the CLI accepts the flag, exits 0, and the rejection
assertion fails for an environment-dependent reason.
Force the gate off explicitly via env { ENABLE_GATED_FEATURES: '' } ('' is
off since isGatedFeaturesEnabled checks === '1'). Verified the block passes
both with the gate unset and with ENABLE_GATED_FEATURES=1 exported.tejaskash
commented
Jun 22, 2026
Good catch — verified and fixed in 26bb9e9. You're right on all three points: I went with option 1 (force the gate off for that one case via Verified the memory-modes block passes both with the gate unset and with |
Claude Security Review: no high-confidence findings. (run) |
Uh oh!
There was an error while loading. Please reload this page.
Summary
Expands the harness integration test suite (
integ-tests/add-remove-harness.test.ts) to cover the harness configuration surface that was previously untested at the integration layer. Follow-up to #1598, which made harness the defaultcreatepath and brought the harness flows into the GA build.All new cases are local — no AWS. They exercise the CLI flag → action → validate → spec-write wiring that schema unit tests don't reach.
Coverage added
lite_llm(+--api-base/--additional-params), bedrock--api-format, plus api-format provider-gating rejections (open_ai/gemini)add tool):remote_mcp,agentcore_code_interpreter,agentcore_gateway(awsIam),inline_function, plus required-config rejectionsadd skill): path / s3 / git (+auth credential)Notes
createspawn instead of one per block) and rejection groups useit.eachtables. EachrunCLIis a real ~1.3s process spawn, so the suite is kept tight.src/schema/schemas/primitives/__tests__/harness.test.ts(top-k / api-base provider gating, skill URI-scheme rejection) are intentionally not re-tested here.fetch access) — tracked separately.Testing
vitest run --project integ integ-tests/add-remove-harness.test.ts→ 35 passed (16 existing + 19 new).tsc --noEmitclean.