Skip to content

direct: wildcard-match UC managed property backend defaults - #5877

Merged
radakam merged 5 commits into
mainfrom
wildcard-uc-managed-property-defaults
Jul 13, 2026
Merged

direct: wildcard-match UC managed property backend defaults#5877
radakam merged 5 commits into
mainfrom
wildcard-uc-managed-property-defaults

Conversation

@radakam

Copy link
Copy Markdown
Contributor

Summary

Use a single wildcard backend-default rule for UC-managed catalog/schema properties: properties['unity.catalog.managed.*.defaults.*']. This replaces per-key entries and covers current and future Auto Upgrade defaults without listing each property.

Why

UC fills these keys after create. If we don't treat them as backend defaults, plans keep trying empty updates and UC rejects them.

@radakam
radakamforce-pushed the wildcard-uc-managed-property-defaults branch from 4db5696 to ad7c34fCompareJuly 9, 2026 10:58
@radakam
radakamtemporarily deployed to test-trigger-is July 9, 2026 11:00 — with GitHub Actions Inactive
@radakam
radakamtemporarily deployed to test-trigger-is July 9, 2026 11:00 — with GitHub Actions Inactive
@radakam
radakam marked this pull request as ready for review July 9, 2026 11:02
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: bf06876

Run: 29238695571

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux4423010815:12
💚​aws windows4423210796:52
💚​aws-ucws linux443169986:20
💚​aws-ucws windows443189968:31
🔄​azure linux22423010806:43
💚​azure windows4423210787:05
💚​azure-ucws linux443189956:35
🔄​azure-ucws windows2243209939:56
🔄​gcp linux13422910827:44
🔄​gcp windows13423110807:50
8 interesting tests: 4 SKIP, 3 flaky, 1 RECOVERED
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R🔄​f🔄​f
🔄​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f💚​R💚​R
🔄​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f💚​R💚​R
Top 10 slowest tests (at least 2 minutes):
durationenvtestname
6:39aws-ucws windowsTestAccept
5:56azure-ucws windowsTestAccept
5:53azure windowsTestAccept
5:53aws windowsTestAccept
5:52gcp windowsTestAccept
2:55gcp linuxTestAccept
2:53azure linuxTestAccept
2:51aws linuxTestAccept
2:46azure-ucws linuxTestAccept
2:43aws-ucws linuxTestAccept

Comment threadNEXT_CHANGELOG.md Outdated
Comment threadlibs/structs/structpath/path.go Outdated
Comment threadlibs/structs/structpath/path.go Outdated
@radakam
radakamforce-pushed the wildcard-uc-managed-property-defaults branch from ad7c34f to c68a120CompareJuly 13, 2026 07:44
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 07:45 — with GitHub Actions Inactive
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 07:45 — with GitHub Actions Inactive
Comment threadlibs/structs/structpath/path_test.go Outdated
Comment threadlibs/structs/structpath/path_test.go Outdated
Replace per-key backend_defaults entries for catalogs and schemas with a
single unity.catalog.managed.*.defaults.* pattern so new Auto Upgrade keys
are recognized without listing each one explicitly.
@radakam
radakamforce-pushed the wildcard-uc-managed-property-defaults branch from c68a120 to 2323babCompareJuly 13, 2026 08:18
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 08:19 — with GitHub Actions Inactive
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 08:19 — with GitHub Actions Inactive

@janniklasrosejanniklasrose left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some suggestions to keep the tests focused

Comment threadlibs/structs/structpath/path_test.go Outdated
Comment threadlibs/structs/structpath/path_test.go Outdated
Comment threadlibs/structs/structpath/path_test.go Outdated
radakamand others added 3 commits July 13, 2026 10:47
Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 08:48 — with GitHub Actions Inactive
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 08:48 — with GitHub Actions Inactive
@radakam
radakam enabled auto-merge July 13, 2026 08:48
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 09:20 — with GitHub Actions Inactive
@radakam
radakamtemporarily deployed to test-trigger-is July 13, 2026 09:20 — with GitHub Actions Inactive
@radakam
radakam added this pull request to the merge queueJul 13, 2026
Merged via the queue into main with commit c9a8fb0Jul 13, 2026
24 checks passed
@radakam
radakam deleted the wildcard-uc-managed-property-defaults branch July 13, 2026 10:02
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: c9a8fb0

Run: 29241341783

Env❌​FAIL🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux44529101040:42
🔄​aws windows244484102355:02
❌​aws-ucws linux111252990820168:30
❌​aws-ucws windows111252932838175:54
💚​azure linux44523101144:13
🔄​azure windows334478102454:34
🔄​azure-ucws linux732894855147:50
💚​azure-ucws windows62840873143:35
💚​gcp linux44514101652:51
🔄​gcp windows134471102961:12
29 interesting tests: 13 flaky, 11 FAIL, 2 RECOVERED, 2 SKIP, 1 KNOWN
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
🟨​TestAccept💚​R💚​R🟨​K🟨​K💚​R🔄​f💚​R💚​R💚​R💚​R
🔄​TestAccept/bundle/deploy/spark-jar-task✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
💚​TestAccept/bundle/invariant/no_drift🙈​S🙈​S💚​R💚​R🙈​S🙈​S💚​R💚​R🙈​S🙈​S
🔄​TestAccept/bundle/resources/clusters/resize-terminated-fallback✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/dashboards/change-name✅​p✅​p✅​p🔄​f✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestAccept/bundle/resources/dashboards/change-name/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p🔄​f✅​p✅​p🔄​f✅​p✅​p✅​p
❌​TestAccept/bundle/resources/postgres_databases/live_errors/missing_role🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_databases/live_errors/missing_role/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/basic🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
💚​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S💚​R💚​R🙈​S🙈​S💚​R💚​R🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=no/READPLAN=✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=yes/READPLAN=1✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f
🔄​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R💚​R
🔄​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R💚​R
Top 50 slowest tests (at least 2 minutes):
durationenvtestname
15:32aws-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:29gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:57gcp linuxTestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
12:51aws-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
12:38gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:05gcp linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:34gcp windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:26gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
10:19azure linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
9:21azure linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
9:17azure-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
9:16azure-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
8:35aws-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:30aws-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:10azure windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
7:52gcp windowsTestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
7:43aws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:39azure-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:36azure windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:27aws-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:18gcp linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
7:11gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
7:00gcp linuxTestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
6:59gcp windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:58aws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:48gcp windowsTestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:39azure-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:35azure windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:24azure-ucws windowsTestAccept
6:24aws-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
6:22azure-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:14azure-ucws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:11gcp windowsTestAccept
6:04azure-ucws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:02azure linuxTestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
5:55azure-ucws windowsTestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:55aws-ucws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
5:50aws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:42aws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:29aws-ucws linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:22azure-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:18azure-ucws linuxTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes/READPLAN=
4:57gcp windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:48gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:45azure-ucws windowsTestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes/READPLAN=
4:40azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:40azure-ucws windowsTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=no/READPLAN=
4:38azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:38aws-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:33aws-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct

deco-sdk-taggingBot added a commit that referenced this pull request Jul 15, 2026
## Release v1.8.0
### Notable Changes
* Auto-migrate a bundle from terraform to the direct engine when `bundle.engine` is `"direct"` (or `DATABRICKS_BUNDLE_ENGINE=direct`) and the post-deploy dry-run migration is clean; a warning is emitted if the dry-run surfaces errors or warnings so the automatic migration is skipped.
### CLI
* experimental `ssh connect`: bare `python`/`pip` in an interactive session now resolve to the environment interpreter (`$DATABRICKS_VIRTUAL_ENV`) instead of the system or cluster-libraries interpreter, so packages installed in the environment are importable without extra setup. The interactive shell is now non-login (`bash -i`) and the server seeds a `~/.bashrc` snippet that re-prepends the environment's bin directory to `PATH` ([#5888](#5888)).
* When Claude Code runs the CLI without the Databricks AI tooling installed, the CLI now prints a one-line recommendation on stderr to run `databricks aitools install`. The recommendation is shown at most once per hour per Claude session, and never for human callers or `aitools` commands.
* Fixed `databricks auth describe` misattributing a profile selected via `DATABRICKS_CONFIG_PROFILE` as `(from bundle)` when run inside a bundle root ([#5904](#5904)).
### Bundles
* `bundle generate` now warns when the generated configuration file is not matched by any pattern in the `include` section of `databricks.yml` ([#5868](#5868)).
* direct: Match UC Auto Upgrade managed property defaults with a wildcard pattern instead of enumerating each key ([#5877](#5877)).
* Recognize `ssh://` template URLs in `databricks bundle init` ([#5891](#5891)).
* `databricks bundle init` now reports an actionable error when given a template URL with an unsupported protocol (`http://`, `git://`, `ftp://`, `ftps://`) instead of failing with a confusing "not a bundle template" message ([#5902](#5902)).
* Added an `env:` section to `scripts.<name>` for declaring environment variables that may reference `${bundle.*}`, `${workspace.*}`, and `${var.*}` ([#4179](#4179), [#5299](#5299)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@radakam@eng-dev-ecosystem-bot@janniklasrose