Skip to content

direct: job with a table_update trigger never converges — trigger.table_update.condition is input-only and not registered as such #6315

Description

@ajheunis

Summary

On the direct deployment engine, a job with a table_update trigger that sets
condition never converges. Every bundle plan after a successful
bundle deploy reports 1 to change, forever.

The Jobs API accepts trigger.table_update.condition on create/update but never
returns it from GET /api/2.2/jobs/get. condition is therefore an input-only
field, and it is not registered as one, so the direct engine diffs it against a
missing remote value and plans an update on every run.

This is the same class of bug as #6076 (allow_duplicate_names on pipelines)
and the webhook_notifications.on_* phantom diff fixed in #6060. It is not
fixed in v1.11.0.

CLI version

v1.10.0 (also reproduces the same way before a resource rename, so it is not
name-related).

Repro

resources:
jobs:
render_files:
name: render_filestrigger:
table_update:
table_names:
- main.default.some_tablecondition: ANY_UPDATEDwait_after_last_change_seconds: 60tasks:
- task_key: tnotebook_task:
notebook_path: ../src/nb.ipynbenvironment_key: defaultenvironments:
- environment_key: defaultspec:
environment_version: "4"
databricks bundle deploy -t dev # succeeds
databricks bundle plan -t dev # Plan: 0 to add, 1 to change, 0 to delete
databricks bundle deploy -t dev # succeeds
databricks bundle plan -t dev # Plan: 0 to add, 1 to change, 0 to delete

Evidence

bundle plan -t dev --output json, plan["resources.jobs.render_files"]:

"changes": {
"trigger.table_update.condition": {
"action": "update",
"old": "ANY_UPDATED",
"new": "ANY_UPDATED"
}
}

old and new are identical and the action is still update. old comes from
the bundle's own saved state (state/resources.json), which records:

"trigger": {
"pause_status": "PAUSED",
"table_update": {
"condition": "ANY_UPDATED",
"table_names": ["<catalog>.<schema>.matched_files"],
"wait_after_last_change_seconds": 60
}
}

while remote_state in the same plan — and the raw
GET /api/2.2/jobs/get?job_id=... — has no condition at all:

"trigger": {
"pause_status": "PAUSED",
"table_update": {
"table_names": ["<catalog>.<schema>.matched_files"],
"wait_after_last_change_seconds": 60
}
}

Two sibling jobs in the same bundle with file_arrival triggers, and a pipeline,
all report as unchanged; only the job with table_update + condition drifts.

Expected

trigger.table_update.condition should be treated as an input-only field so the
plan converges, the same way #6076 handled allow_duplicate_names.

Workaround

Omit condition from the bundle config. Where a trigger names a single table,
ANY_UPDATED and ALL_UPDATED are equivalent, so this costs nothing there — but
it is not an option for a multi-table trigger that needs ALL_UPDATED.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions