Skip to content

Add automations diff to preview a spec against the live automation - #12

Merged
jbedient-kizen merged 2 commits into
bug/automation-update-deactivates-without-askingfrom
feat/automations-diff-spec-vs-live
Aug 17, 2026
Merged

Add automations diff to preview a spec against the live automation#12
jbedient-kizen merged 2 commits into
bug/automation-update-deactivates-without-askingfrom
feat/automations-diff-spec-vs-live

Conversation

@jbedient-kizen

@jbedient-kizen jbedient-kizen commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Stacked on #7. Base is bug/automation-update-deactivates-without-asking
(BCLI-016), not main, because diff resolves active the same tri-state way
that PR introduces. Merge #7 first; this PR's diff shows only its own work.

Problem

automations update was all-or-nothing. To know what a spec would do to a live
automation you either trusted it and wrote, or diffed two full JSON trees by
eye. Neither scales past a handful of steps, and the failure mode is silent:
an unintended reparenting or a dropped step looks like any other successful
write.

Solution

kizen automations diff <api_name> --spec-file <path> (stdin also accepted)
builds the payload an update would PUT and compares it to the live
automation, writing nothing.

The hard part is identity, not comparison. GET and PUT are different dialects,
so key is resynthesized from live order on one side and hand-authored on the
other — compared literally, every step reads as changed and the output is
worthless. So:

  • Steps and triggers are matched by id first, regardless of key or order.
    Position is a fallback only for a spec carrying no ids at all.
  • key/parent_key/prefix are excluded from the field comparison as
    per-side synthetic naming rather than automation content.
  • Reparenting and go_to_automation_step references are resolved to their
    target's matched identity, so a genuine retarget still surfaces while a
    cosmetic rekey does not.

Each line is labelled with the first octet of the step/trigger's id, which
matches what the UI shows and is unique within one automation.

One behaviour decision worth pushing on: a spec item carrying an id that
matches no live step is now treated as an addition, and the live step it would
have displaced as a removal — rather than merged into that step by position.
This is what the PUT would actually do, and the previous position-merge pinned
both sides' id to the live value, silently swallowing the unknown one and
rendering it as "these fields changed."

Testing

check.sh builder-cli fully green: 1128 passed, 4 skipped, 64 deselected;
lint, format, typecheck, and the CLI-tree extra_checks all pass.

Both behavioural fixes were proven by disabling the fix and watching the new
test fail, then restoring it:

  • go_to resolution disabled → test_diff_wire_payloads_ignores_go_to_key_resynthesis
    and test_diff_wire_payloads_reports_go_to_retarget both fail.
  • dangling-id handling reverted → test_diff_wire_payloads_dangling_spec_id_is_addition_not_edit
    fails with a KeyError on the expected removal, while the other nine wire
    tests pass under both behaviours, confirming the change is narrowly scoped.

Also sanity-checked read-only against a live automation in cli-testing:
rebuilding a spec from automations show --json and diffing it returned an
empty diff, with the automation's revision unchanged before and after.

Design notes / tradeoffs

  • active resolution is duplicated rather than shared with
    plan_update_automation, deliberately: sharing it would put diff on that
    function's single-fetch path and cost a second live fetch. Worth extracting
    once both call sites have settled.
  • Field-level --json entries carry only the id octet, not a separate full
    id key. Additions and removals carry the whole step/trigger including its
    full id; every other command's --json (automations get, show) carries
    full ids throughout, so this is scoped to diff's changed-field entries.
    An earlier revision of this note justified that by "keeping the shape
    identical to roundtrip's existing drift field" — the three keys do match,
    but drift numbers steps positionally and carries no id at all, so it is no
    precedent for octet-only identity. What this actually rests on is that the
    octet is unique within one automation, plus not wanting an id key that is
    present on step/trigger entries and absent on top-level automation-field
    entries like {"path": "name"}. That is a weaker guarantee than simply
    including the id — say so and it gets added.
  • --output csv has no id column. Out of scope here; worth a follow-up.

🤖 Generated with Claude Code

jbedient-kizen and others added 2 commits August 13, 2026 14:17
`automations update` was all-or-nothing: you either trusted the spec or
eyeballed two full JSON trees. `diff` shows what the update would change
— trigger/step additions, removals, reparenting, and field changes.

The hard part is identity. GET and PUT are different dialects, so `key`
is resynthesized from live order on one side and hand-authored on the
other; comparing them literally reports every step as changed. Steps and
triggers are matched by `id`, with `key`/`parent_key`/`prefix` excluded
as per-side naming rather than content. Reparenting and
`go_to_automation_step` references are resolved to the matched identity
of their target, so they still surface a real change without tripping on
a cosmetic rekey.

A spec item carrying an `id` that matches no live step is treated as an
addition, and the live step it would have displaced as a removal, rather
than merged into it by position — that is what the PUT would actually
do, and the previous position-merge silently swallowed the unknown id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`diff_wire_payloads`'s docstring said its return value was "the same shape
`roundtrip_automation`'s `drift` field already uses." The three keys match;
the `path` convention does not. `drift` comes from `semantic_diff`, which
identifies steps positionally (`steps[3].field`) and carries no id anywhere,
so it is not a precedent for these id-octet-labelled paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jbedient-kizen
jbedient-kizen merged commit 6c10431 into bug/automation-update-deactivates-without-asking Aug 17, 2026
4 checks passed
@jbedient-kizen
jbedient-kizen deleted the feat/automations-diff-spec-vs-live branch August 17, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants