Summary
On the direct deployment engine, a job with a table_update trigger that sets
condition never converges. Every bundle plan after a successful
bundle deploy reports 1 to change, forever.
The Jobs API accepts trigger.table_update.condition on create/update but never
returns it from GET /api/2.2/jobs/get. condition is therefore an input-only
field, and it is not registered as one, so the direct engine diffs it against a
missing remote value and plans an update on every run.
This is the same class of bug as #6076 (allow_duplicate_names on pipelines)
and the webhook_notifications.on_* phantom diff fixed in #6060. It is not
fixed in v1.11.0.
CLI version
v1.10.0 (also reproduces the same way before a resource rename, so it is not
name-related).
Repro
resources:
jobs:
render_files:
name: render_filestrigger:
table_update:
table_names:
- main.default.some_tablecondition: ANY_UPDATEDwait_after_last_change_seconds: 60tasks:
- task_key: tnotebook_task:
notebook_path: ../src/nb.ipynbenvironment_key: defaultenvironments:
- environment_key: defaultspec:
environment_version: "4"
databricks bundle deploy -t dev # succeeds
databricks bundle plan -t dev # Plan: 0 to add, 1 to change, 0 to delete
databricks bundle deploy -t dev # succeeds
databricks bundle plan -t dev # Plan: 0 to add, 1 to change, 0 to delete
Evidence
bundle plan -t dev --output json, plan["resources.jobs.render_files"]:
"changes": {
"trigger.table_update.condition": {
"action": "update",
"old": "ANY_UPDATED",
"new": "ANY_UPDATED"
}
}old and new are identical and the action is still update. old comes from
the bundle's own saved state (state/resources.json), which records:
"trigger": {
"pause_status": "PAUSED",
"table_update": {
"condition": "ANY_UPDATED",
"table_names": ["<catalog>.<schema>.matched_files"],
"wait_after_last_change_seconds": 60
}
}while remote_state in the same plan — and the raw
GET /api/2.2/jobs/get?job_id=... — has no condition at all:
"trigger": {
"pause_status": "PAUSED",
"table_update": {
"table_names": ["<catalog>.<schema>.matched_files"],
"wait_after_last_change_seconds": 60
}
}Two sibling jobs in the same bundle with file_arrival triggers, and a pipeline,
all report as unchanged; only the job with table_update + condition drifts.
Expected
trigger.table_update.condition should be treated as an input-only field so the
plan converges, the same way #6076 handled allow_duplicate_names.
Workaround
Omit condition from the bundle config. Where a trigger names a single table,
ANY_UPDATED and ALL_UPDATED are equivalent, so this costs nothing there — but
it is not an option for a multi-table trigger that needs ALL_UPDATED.
Summary
On the direct deployment engine, a job with a
table_updatetrigger that setsconditionnever converges. Everybundle planafter a successfulbundle deployreports1 to change, forever.The Jobs API accepts
trigger.table_update.conditionon create/update but neverreturns it from
GET /api/2.2/jobs/get.conditionis therefore an input-onlyfield, and it is not registered as one, so the direct engine diffs it against a
missing remote value and plans an update on every run.
This is the same class of bug as #6076 (
allow_duplicate_nameson pipelines)and the
webhook_notifications.on_*phantom diff fixed in #6060. It is notfixed in v1.11.0.
CLI version
v1.10.0 (also reproduces the same way before a resource rename, so it is not
name-related).
Repro
Evidence
bundle plan -t dev --output json,plan["resources.jobs.render_files"]:oldandneware identical and the action is stillupdate.oldcomes fromthe bundle's own saved state (
state/resources.json), which records:while
remote_statein the same plan — and the rawGET /api/2.2/jobs/get?job_id=...— has noconditionat all:Two sibling jobs in the same bundle with
file_arrivaltriggers, and a pipeline,all report as unchanged; only the job with
table_update+conditiondrifts.Expected
trigger.table_update.conditionshould be treated as an input-only field so theplan converges, the same way #6076 handled
allow_duplicate_names.Workaround
Omit
conditionfrom the bundle config. Where a trigger names a single table,ANY_UPDATEDandALL_UPDATEDare equivalent, so this costs nothing there — butit is not an option for a multi-table trigger that needs
ALL_UPDATED.