Uh oh!
There was an error while loading. Please reload this page.
Add real-n8n e2e test suite, bug fixes, and CI hardening - #1
Merged
Conversation
- folder commands wrap their actions in runFolder() so internal-API errors
return {error,code} instead of leaking a raw ApiRequestError stack trace
- wf publish no longer sends the read-only `id` on create (n8n rejects it);
--dry create now reports a provisional null id
- table formatter sets colWidths only when defined, fixing a cli-table3 crash
on --table for commands without explicit column widths
Found and reproduced against live n8n 2.25.7 by the new e2e suite.- Testcontainers-backed n8n (tag+digest pinned, ephemeral SQLite); globalSetup mints a full-scope API key and provides creds to specs via vitest inject - 51 e2e specs across every command group (happy / 404 / 401 / --dry / --table / golden snapshot), a macOS real-keychain spec, and a 429 backoff unit test - helpers: typed json<T>() accessor, toFailWithCode matcher, redaction with a volatile-leak guard, per-test onTestFinished cleanup, env-allowlist spawning - add test:e2e / test:e2e:macos scripts; bump engines.node to >=22.22
- ubuntu e2e job with a digest-verified n8n image cache, plus a macOS keychain job (no Docker); both required, Node 22 - SHA-pin actions, least-privilege permissions, per-job timeouts, PR-scoped concurrency - add Dependabot for github-actions and npm
listDataTableRows paginates through every page, so `--limit` (advertised as the max rows to return) had no effect on the total. Slice the merged result to the limit. Surfaced by the new dt rows e2e coverage.
The login subcommand declared its own required `--api-key`, which the global `--api-key` option shadowed, so `auth login` could never satisfy it. Read the key from the global flag instead (use "-" for stdin). Surfaced by the new keychain e2e coverage.
- unit: BaseClient.paginateAll cursor loop (the multi-page path behind every list) - wf publish create/dry/no-files, wf save all, folder sync/move, dt malformed input + --limit cap + table formatter, proj update, sc pull --force - auth: 401 wrong-key contract (e2e) and login/set-credentials keychain round-trips - JSON golden snapshots for user get, dt get, wf list item - test/e2e/COVERAGE.md: command -> branch -> spec map with deferral rationale
`docker save`/`load` of a digest-only ref does not preserve the RepoDigest, so on a cache hit the digest-verification step failed with "No such object". Drop the tar cache and pre-pull the exact tag+digest ref; the registry verifies the digest and Testcontainers reuses the local image.
Uh oh!
There was an error while loading. Please reload this page.
marcinobel added a commit
that referenced
this pull request
Jun 18, 2026
Lens-review remediation (round 2). Security / supply chain: - publish.yml: pin `npm@11` instead of mutable `npm@latest` in the OIDC publish job (#2); add `environment: release` to gate the token mint behind a protected environment (#3 — configure required reviewer + v* tag filter in repo settings for it to take effect). - ci.yml: pin `reuse==6.2.0` for the REUSE lint step (reproducibility, #13). Docs: - CLA.md / CONTRIBUTING.md / README: drop the claim that a CLA-assistant bot auto-posts a signing link (none is installed); opening a PR is the agreement, bot may be added later (#1). - README: normalize the new prose to en dashes (house style, #11).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an end-to-end test suite that spawns the built CLI (
dist/bin/8cli.js) against a real n8n started by Testcontainers, covering every command group. Includes a macOS real-keychain spec, a 429-backoff unit test, three production bug-fixes the suite surfaced, and hardening of the new CI.Bugs found & fixed (against live n8n 2.25.7)
ApiRequestErrorstack trace instead of the structured{error,code}JSON — now wrapped inrunFolder().wf publishcouldn't create workflows — it sent a read-onlyidn8n rejects; create now sends only the stripped payload (--dryreports a provisionalnullid).--tablecrashed (cli-table3colWidths: undefined) for any command without explicit column widths — affected tag/credential/variable/project/user/datatable.Test design
globalSetupbootstraps an owner, mints a full-scope API key, and provides creds to specs via vitestinject. Readiness-gated on/healthz/readiness.--dry/--table/ golden snapshot; stdin &@fileinputs;wf diffboth branches; PUT-gotcha stripping; transfer/idempotent flows; transport failure; config precedence. License-gated groups (variables, projects, folders, source-control) assert the gated-error contract — the suite targets free Community n8n, no license.json<T>()accessor, atoFailWithCodematcher (exit 1 + structured error + no stack-trace leak), key-based snapshot redaction with a volatile-leak guard, per-testonTestFinishedcleanup, env-allowlist spawning.CI
e2e(ubuntu, digest-verified n8n image cache) ande2e-macos(no Docker, local HTTP mock) jobs — both required, Node 22.permissions: {}+ per-job scope, job timeouts, PR-scoped concurrency, Dependabot.Verification
npm run test:e2e(51) ·npm run test:e2e:macos(2) ·npm test(17) all green; typecheck, lint, format, header check, build,npm audit, andactionlintclean; snapshot stable across runs.