Skip to content

Replace server_side_default with backend_default - #4490

Merged
denik merged 21 commits into
mainfrom
denik/backend_defaults
Feb 18, 2026
Merged

Replace server_side_default with backend_default#4490
denik merged 21 commits into
mainfrom
denik/backend_defaults

Conversation

@denik

@denikdenik commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Add new backend_default setting in resources.yml that allows configuring known server defaults for given field patterns.
  • Port config for backend_default and ignore_remote_changes from terraform.

Why

The "server_side_default" assumes that any setting appearing in the remote state that is not in the config should be ignored. This works when resources are only updated via DABs "bundle deploy" but it can miss real drift.

Tests

Existing tests.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Feb 11, 2026

Copy link
Copy Markdown
Collaborator

Commit: aa91788

Run: 22140635452

Env🟨​KNOWN💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
🟨​aws linux7172687636:48
🟨​aws windows7172707615:54
💚​aws-ucws linux873646796:59
💚​aws-ucws windows873666775:27
💚​azure linux292717615:53
💚​azure windows292737594:31
💚​azure-ucws linux293696757:29
💚​azure-ucws windows293716735:41
💚​gcp linux292677645:24
💚​gcp windows292697624:59
15 interesting tests: 7 KNOWN, 7 SKIP, 1 RECOVERED
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
🟨​TestAccept🟨​K🟨​K💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/permissions🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions🟨​K🟨​K💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions🟨​K🟨​K💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform🟨​K🟨​K💚​R💚​R
🙈​TestAccept/bundle/resources/postgres_branches/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/update_protected🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/without_branch_id🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_endpoints/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/synced_database_tables/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/ssh/connection💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
Top 20 slowest tests (at least 2 minutes):
durationenvtestname
3:46gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:13gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:12gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:06gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:00aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:49aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:46aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:43aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:42aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:41aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:41aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:17azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:13azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:11azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:11azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:11azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:10azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:08azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:05azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@denik
denikforce-pushed the denik/patterns-resource-config branch from 70c35f1 to 2fb9d43CompareFebruary 11, 2026 12:03
Base automatically changed from denik/patterns-resource-config to mainFebruary 11, 2026 12:33
@denik
denikforce-pushed the denik/backend_defaults branch from 88e8d44 to 4aaf27cCompareFebruary 11, 2026 12:54
@denik
denikforce-pushed the denik/backend_defaults branch from 4aaf27c to 3dc1550CompareFebruary 11, 2026 12:55
@denik
denik changed the base branch from main to denik/emptyFebruary 11, 2026 12:59
denik added a commit that referenced this pull request Feb 11, 2026
…#4491)
## Changes
New "empty" reason in plan triggers when old/new & remote values are all
either nil or zero or empty slice/map.
## Why
Ignoring differences like nil vs "" and nil vs 0 reduces drift when
backend has preferred representation. For example,
model_serving_endpoints.description is converted from nil to "" by
backend. jobs.timeout_seconds has a default 0.
This supports removing server_side_default rule which currently marks
this changes as "skip". #4490
## Tests
Existing tests.
Base automatically changed from denik/empty to mainFebruary 11, 2026 15:12
@denik
denikforce-pushed the denik/backend_defaults branch from e2ec42e to 7626cbdCompareFebruary 11, 2026 15:13
@denik
denikforce-pushed the denik/backend_defaults branch from 8795ceb to 614ee77CompareFebruary 12, 2026 10:47
@denik
denikforce-pushed the denik/backend_defaults branch from 614ee77 to c31e636CompareFebruary 12, 2026 11:07
@denik
denikforce-pushed the denik/backend_defaults branch from c31e636 to 2d50de6CompareFebruary 18, 2026 11:08
@denik
denik marked this pull request as ready for review February 18, 2026 11:31
denikand others added 12 commits February 18, 2026 13:21
…aults
Cross-referenced terraform-provider-databricks source (SetComputed,
SetSuppressDiff, SetForceNew) against resources.yml to fix gaps.
Added backend_defaults (SetComputed/SetSuppressDiff — backend-set fields):
- clusters: enable_local_disk_encryption, node_type_id, driver_node_type_id,
driver_instance_pool_id
- jobs: run_as, spark_python_task.source, sql_task.file.source,
dbt_task.source (+ for_each_task variants for all), new_cluster computed
fields matching clusters
- pipelines: event_log.catalog, event_log.schema, cluster computed fields
- sql_warehouses: enable_serverless_compute
- experiments: artifact_location
Added ignore_remote_changes:
- sql_warehouses: min_num_clusters (SetSuppressDiff)
- quality_monitors: skip_builtin_dashboard (not returned by API)
Added recreate_on_changes:
- pipelines: gateway_definition.{connection_id,connection_name,
gateway_storage_catalog,gateway_storage_schema},
ingestion_definition.ingest_from_uc_foreign_catalog
Removed dead backend_defaults — these are TF client-side SetDefault()
values already applied by resource_mutator.go, so the field is never nil
and backend_defaults (which triggers on old=nil AND new=nil) never fires:
- jobs: max_concurrent_runs, schedule/trigger/continuous.pause_status
- pipelines: edition, channel
- dashboards: embed_credentials (also had wrong value: yml said true,
mutator sets false)
- clusters: autotermination_minutes
- sql_warehouses: auto_stop_mins, enable_photon, max_num_clusters,
spot_instance_policy
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@denik
denikforce-pushed the denik/backend_defaults branch from 4611e58 to dfe0736CompareFebruary 18, 2026 12:42
# skip_builtin_dashboard is input-only, not returned by the Get API.
# TF preserves it from state; see resource_quality_monitor.go.
- field: skip_builtin_dashboard
reason: not_returned_by_api

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not ignore_remote_changes with reason: input_only?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, not sure why have not_returned_by_api at all. Replaced.

~ edition: null -> "ADVANCED"
~ libraries: [{"glob":{"include":"/Workspace/Users/[USERNAME]/lakeflow_pipeline/transformations/**"}},{"glob":{"include":"/Workspace/Users/foo@databricks.com/another/**"}}] -> [{"notebook":{"path":"/Workspace/Users/[USERNAME]/.bundle/test-pipeline-recreate/default/files/nb"}}]
~ name: "lakeflow-pipeline" -> "test-pipeline"
~ root_path: "/Workspace/Users/[USERNAME]/lakeflow_pipeline" -> null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happened here?

@denikdenikFeb 18, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is real change in this test (it was set on remote but not in the config), but it was NOT noticed before because server_side_default applied.

@denik
denik enabled auto-merge February 18, 2026 13:00
@denik
denik added this pull request to the merge queueFeb 18, 2026
Merged via the queue into main with commit 1813c28Feb 18, 2026
18 checks passed
@denik
denik deleted the denik/backend_defaults branch February 18, 2026 13:38
deco-sdk-taggingBot added a commit that referenced this pull request Feb 19, 2026
## Release v0.289.0
### CLI
* Make auth profiles respect DATABRICKS_CLI_PATH env var (([#4467](#4467)))
* Fix arrow key navigation in prompts on Windows (([#4501](#4501)))
### Bundles
* Log artifact build output in debug mode ([#4208](#4208))
* Fix bundle init not working in Azure Government ([#4286](#4286))
* Allow single and double quotes in environment dependencies (([#4511](#4511)))
* Use purge option when deleting alerts (([#4505](#4505)))
* engine/direct: Replace server_side_default with more precise backend_default rule in bundle plan ([#4490](#4490))
* engine/direct: Extend pipelines recreate_on_changes configuration (([#4499](#4499)))
* engine/direct: Added support for UC external locations (direct only) ([#4484](#4484))
### Dependency updates
* Upgrade Go SDK to v0.106.0 (([#4486](#4486)))
* Upgrade Terraform provider to v1.106.0 (([#4542](#4542)))
* Upgrade Go to 1.25.7 (([#4547](#4547)))
denik added a commit that referenced this pull request May 20, 2026
…#4491)
## Changes
New "empty" reason in plan triggers when old/new & remote values are all
either nil or zero or empty slice/map.
## Why
Ignoring differences like nil vs "" and nil vs 0 reduces drift when
backend has preferred representation. For example,
model_serving_endpoints.description is converted from nil to "" by
backend. jobs.timeout_seconds has a default 0.
This supports removing server_side_default rule which currently marks
this changes as "skip". #4490
## Tests
Existing tests.
denik added a commit that referenced this pull request May 20, 2026
## Changes
- Add new backend_default setting in resources.yml that allows
configuring known server defaults for given field patterns.
- Port config for backend_default and ignore_remote_changes from
terraform.
## Why
The "server_side_default" assumes that any setting appearing in the
remote state that is not in the config should be ignored. This works
when resources are only updated via DABs "bundle deploy" but it can miss
real drift.
## Tests
Existing tests.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
denik pushed a commit that referenced this pull request May 20, 2026
## Release v0.289.0
### CLI
* Make auth profiles respect DATABRICKS_CLI_PATH env var (([#4467](#4467)))
* Fix arrow key navigation in prompts on Windows (([#4501](#4501)))
### Bundles
* Log artifact build output in debug mode ([#4208](#4208))
* Fix bundle init not working in Azure Government ([#4286](#4286))
* Allow single and double quotes in environment dependencies (([#4511](#4511)))
* Use purge option when deleting alerts (([#4505](#4505)))
* engine/direct: Replace server_side_default with more precise backend_default rule in bundle plan ([#4490](#4490))
* engine/direct: Extend pipelines recreate_on_changes configuration (([#4499](#4499)))
* engine/direct: Added support for UC external locations (direct only) ([#4484](#4484))
### Dependency updates
* Upgrade Go SDK to v0.106.0 (([#4486](#4486)))
* Upgrade Terraform provider to v1.106.0 (([#4542](#4542)))
* Upgrade Go to 1.25.7 (([#4547](#4547)))
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@denik@eng-dev-ecosystem-bot@pietern@ilyakuz-db