Skip to content

direct: Record the CLI that last wrote the deployment state - #6222

Merged
denik merged 2 commits into
mainfrom
denik/state_cli_version
Aug 12, 2026
Merged

direct: Record the CLI that last wrote the deployment state#6222
denik merged 2 commits into
mainfrom
denik/state_cli_version

Conversation

@denik

Copy link
Copy Markdown
Contributor

Why

cli_version in resources.json kept the version of the CLI that created the state. Every deploy
wrote the current version into the WAL header, but replay copied only lineage and serial back into
the state, so the field never moved — a state deployed daily by the latest CLI still reported
whatever version first created it.

The terraform engine updates its own cli_version on every deploy, and the field is documented as
the version that last wrote the state, so this brings the direct engine in line with both.

It is refreshed alongside the serial, under the same condition: the state file is only persisted when
the WAL carried entries, so a deploy that commits nothing must not claim to have written one.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 4b2f17d

Run: 31491868526

Env❌​FAIL🟨​KNOWN💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
❌​aws linux113429411309:16
❌​aws windows1134296112810:08
❌​azure linux113429311308:46
❌​azure windows1134295112810:02
❌​gcp linux11529411308:18
❌​gcp windows11529611289:16
9 interesting tests: 4 SKIP, 3 RECOVERED, 1 KNOWN, 1 FAIL
Test Nameaws linuxaws windowsazure linuxazure windowsgcp linuxgcp windows
🟨​TestAccept🟨​K🟨​K🟨​K🟨​K🟨​K🟨​K
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
❌​TestAccept/pipelines/destroy/cascade-on-destroy-terraform-error❌​F❌​F❌​F❌​F❌​F❌​F
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R

cli_version in resources.json kept the version of the CLI that created the
state: every deploy wrote the current version into the WAL header, but
replay copied only lineage and serial back, so the field never moved.
Refresh it alongside the serial, which is the same condition (the state
file is only persisted when the WAL carried entries). This matches the
terraform engine, which updates its own cli_version on every deploy, and
the field's documented meaning.
@denik
denikforce-pushed the denik/state_cli_version branch from 4f78708 to 4b2f17dCompareAugust 11, 2026 12:33
@denik
denik enabled auto-merge August 11, 2026 12:55
@denik
denik disabled auto-merge August 12, 2026 10:15
@denik
denik merged commit 9d5fd4e into mainAug 12, 2026
9 checks passed
@denik
denik deleted the denik/state_cli_version branch August 12, 2026 10:15
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 9d5fd4e

Run: 31586747155

Env❌​FAIL🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
❌​aws linux614521157940204:35
❌​aws windows612521098959246:20
🔄​azure linux4621057977196:13
🔄​azure windows462996996234:14
💚​gcp linux331051981212:11
🔄​gcp windows6339841000229:53
34 interesting tests: 20 flaky, 6 FAIL, 5 RECOVERED, 2 SKIP, 1 KNOWN
Test Nameaws linuxaws windowsazure linuxazure windowsgcp linuxgcp windows
🟨​TestAccept🟨​K🟨​K💚​R💚​R💚​R💚​R
🔄​TestAccept/bundle/deploy/snapshot-comparison✅​p✅​p✅​p🔄​f✅​p✅​p
🔄​TestAccept/bundle/deploy/snapshot-comparison/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p🔄​f✅​p✅​p
🔄​TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema.yml.tmpl/READPLAN=✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema_uppercase_name.yml.tmpl/READPLAN=✅​p🔄​f✅​p✅​p✅​p✅​p
💚​TestAccept/bundle/invariant/no_drift💚​R💚​R💚​R💚​R💚​R💚​R
🔄​TestAccept/bundle/resources/dashboards/dataset-catalog-schema✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/dashboards/dataset-catalog-schema/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/grants/registered_models✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/grants/registered_models/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/grants/schemas/all_privileges_coexist✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/resources/grants/schemas/all_privileges_coexist/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/resources/model_serving_endpoints/basic✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p✅​p✅​p🔄​f
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions❌​F❌​F🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions❌​F❌​F🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
🔄​TestAccept/bundle/resources/postgres_projects/update_display_name🔄​f✅​p🙈​s🙈​s🙈​s🙈​s
🔄​TestAccept/bundle/resources/postgres_projects/update_display_name/DATABRICKS_BUNDLE_ENGINE=direct🔄​f✅​p
🔄​TestAccept/bundle/resources/quality_monitors/change_table_name✅​p✅​p✅​p🔄​f✅​p✅​p
🔄​TestAccept/bundle/resources/quality_monitors/change_table_name/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p🔄​f✅​p✅​p
💚​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestAccept/bundle/templates/default-python/combinations/serverless✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=yes/PY=yes/READPLAN=✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/templates/default-python/integration_classic🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13🔄​f✅​p✅​p✅​p✅​p✅​p
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R
Top 50 slowest tests (at least 2 minutes):
durationenvtestname
15:13aws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
15:04gcp linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:50gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:47gcp windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:57aws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:57gcp linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
8:12gcp linuxTestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
8:05azure linuxTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:03azure linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
8:01aws windowsTestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:00gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:57azure windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:57aws windowsTestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:47azure windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:47aws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:45azure linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:38aws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:38azure linuxTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:32gcp linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
7:13azure linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:06aws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:02gcp linuxTestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:29gcp linuxTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
6:29gcp windowsTestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
6:29gcp windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:26aws windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
6:22azure linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:21gcp linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:17aws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:17azure linuxTestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
6:04aws linuxTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=no/PY=yes/READPLAN=1
6:04gcp linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
5:58gcp windowsTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:45aws windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_pydabs_1000_tasks.yml.tmpl/READPLAN=1
5:42aws linuxTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no/READPLAN=
5:41azure windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=model_serving_endpoint.yml.tmpl/READPLAN=
5:40aws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:37azure windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:36aws windowsTestAccept/bundle/resources/postgres_databases/update/DATABRICKS_BUNDLE_ENGINE=terraform
5:35gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
5:33aws windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:30gcp linuxTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:28gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:23azure windowsTestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl
5:22gcp windowsTestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:22gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:20gcp windowsTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
5:19gcp windowsTestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:10gcp windowsTestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=no/READPLAN=
5:09aws windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1

denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@eng-dev-ecosystem-bot@andrewnester