Skip to content

terraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABs - #5621

Merged
denik merged 11 commits into
mainfrom
denik/fix-dabs-to-terraform-path
Jun 23, 2026
Merged

terraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABs#5621
denik merged 11 commits into
mainfrom
denik/fix-dabs-to-terraform-path

Conversation

@denik

@denikdenik commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Make DABsPathToTerraform the exact inverse of TerraformPathToDABs:

  • Prepend the TF wrapper segment (e.g. spec) only for fields that actually live under it, driven by a generated DABsToTerraformWrapperFields set. Root-level fields (status, timestamps, IDs) and unknowns now round-trip correctly.
  • Map the mlflow model's numeric ID across engines: model_id (direct) ↔ registered_model_id (Terraform). It's a state-computed field the codegen heuristics can't derive, so it's wired up via a small manualRenames table; registered_model_id is no longer classified Terraform-only.

A reference to registered_model_id now resolves on both the terraform and direct engines, covered by a new acceptance test.

Needed for #5399.

This pull request and its description were written by Isaac.

@denik
deniktemporarily deployed to test-trigger-is June 16, 2026 18:09 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2026 18:09 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 5b1f10c

Run: 27845764480

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux71326510125:14
💚​aws windows71326710107:02
💚​aws-ucws linux7133619266:16
💚​aws-ucws windows7133639248:02
💚​azure linux11526810104:54
💚​azure windows11527010086:52
💚​azure-ucws linux1153669226:42
🔄​azure-ucws windows21153669209:17
💚​gcp linux11526410135:55
💚​gcp windows11526610117:36
22 interesting tests: 13 SKIP, 7 RECOVERED, 2 flaky
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/permissions🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/postgres_branches/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/replace_existing🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/update_protected🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/without_branch_id🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_endpoints/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_projects/update_display_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/synced_database_tables/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestFetchRepositoryInfoAPI_FromRepo/root✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p
🔄​TestFetchRepositoryInfoAPI_FromRepo/subdir✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p
Top 20 slowest tests (at least 2 minutes):
durationenvtestname
4:45gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:32gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:03gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:59gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:22aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:20azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:18aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:08aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:06aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:03aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:53azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:51azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:48aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:47aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:35aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:33azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:32azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:26azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@denik
denikforce-pushed the denik/fix-dabs-to-terraform-path branch from 4ca0527 to c64a6f3CompareJune 17, 2026 01:14
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 01:15 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 01:15 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 01:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 01:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 21:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 17, 2026 21:24 — with GitHub Actions Inactive
denik added 8 commits June 18, 2026 15:39
…level TF fields
For postgres groups with a spec wrapper, DABsPathToTerraform was
unconditionally prepending "spec" to all paths. Root-level TF fields
(name, status.*, create_time, project_id, etc.) live outside the wrapper
and must not receive the prefix.
Extend the codegen to emit DABsToTerraformRootFields: for each wrapper
group, the set of first-level TF field names that are at the resource
root. DABsPathToTerraform now only prepends the wrapper when the path's
first segment is absent from that set.
Remove noRoundtrip: true from the affected translate_test.go cases; add
new roundtrip cases for project_id and name.
Co-authored-by: Isaac
…st.go
All TerraformPathToDABs test cases now round-trip correctly through
DABsPathToTerraform, so the field and its guard are no longer needed.
Co-authored-by: Isaac
TerraformPathToDABs should be idempotent: a DABs-format path passed
through a second time must come back unchanged. Assert this for every
non-error test case.
Co-authored-by: Isaac
Rebase onto main picked up the new databricks_postgres_role resource;
regenerate generated.go to include it.
Co-authored-by: Isaac
Replace DABsToTerraformRootFields (exception-based: fields NOT under the
wrapper) with DABsToTerraformWrapperFields (positive: fields that ARE under
the wrapper). DABsPathToTerraform now prepends the wrapper only when the
first path segment appears in this set; unknown and root-level fields pass
through unchanged without requiring explicit enumeration.
Co-authored-by: Isaac
@denik
denikforce-pushed the denik/fix-dabs-to-terraform-path branch from 0fdd7cc to f2e6aeeCompareJune 18, 2026 22:41
@denik
deniktemporarily deployed to test-trigger-is June 18, 2026 22:41 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 18, 2026 22:41 — with GitHub Actions Inactive
The mlflow model's numeric ID is a state-computed field named model_id in the
direct engine and registered_model_id in Terraform. It lives in RemoteType, not
the config struct, and the two names are lexically unrelated, so the codegen's
heuristic matcher cannot derive the rename. Add a small manualRenames table so
the rename flows into both translation maps and registered_model_id is no longer
classified Terraform-only.
This lets a reference to registered_model_id resolve on both engines, covered by
a new acceptance test.
Co-authored-by: Isaac
@denik
deniktemporarily deployed to test-trigger-is June 18, 2026 22:48 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 18, 2026 22:48 — with GitHub Actions Inactive
@denikdenik changed the title terraform_dabs_map: fix DABsPathToTerraform to skip wrapper for TF root-level fieldsterraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABsJun 18, 2026
@denik
deniktemporarily deployed to test-trigger-is June 19, 2026 19:46 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 19, 2026 19:46 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 19, 2026 19:56 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 19, 2026 19:56 — with GitHub Actions Inactive
@denik
denik enabled auto-merge June 19, 2026 20:07
Comment threadNEXT_CHANGELOG.md

### Bundles
* `bundle run` now prints the modern job run URL (`/jobs/<id>/runs/<id>`) so that non-admin users permitted to view the run are taken to the run instead of the workspace homepage.
* References to a registered model's `registered_model_id` now resolve under the direct engine, matching Terraform behavior ([#5621](https://github.com/databricks/cli/pull/5621)).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is registered_model_id? I don't see it here in our public documentation:
https://docs.databricks.com/api/workspace/registeredmodels/create

@denikdenikJun 22, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the field that DABs on terraform supports. So we translate it for compat between terraform & direct engine.

@denik
denik added this pull request to the merge queueJun 23, 2026
Merged via the queue into main with commit 633245fJun 23, 2026
29 checks passed
@denik
denik deleted the denik/fix-dabs-to-terraform-path branch June 23, 2026 08:44
denik added a commit that referenced this pull request Jun 23, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.
Co-authored-by: Isaac
denik added a commit that referenced this pull request Jun 24, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.
Co-authored-by: Isaac
denik added a commit that referenced this pull request Jun 24, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.
Co-authored-by: Isaac
deco-sdk-taggingBot added a commit that referenced this pull request Jun 24, 2026
## Release v1.5.0
### CLI
* `workspace export-dir` no longer aborts when a workspace object's name is not a legal local filename (e.g. a notebook named `New Notebook 2026-05-04 13:54:24` whose `:` is illegal on Windows). Such files are now exported under a sanitized name with a warning and the export completes ([#5171](#5171)).
* `ssh connect` now opens an interactive `bash` login shell by default instead of the compute image's default `/bin/sh`, falling back gracefully when `bash` is unavailable. Passing an explicit remote command (`-- <cmd>`) is unaffected ([#5687](#5687)).
* `ssh connect` interactive sessions now start in the user's workspace home folder (`/Workspace/Users/<email>`) instead of the OS home directory, falling back to the OS home when that folder is unavailable ([#5688](#5688)).
### Bundles
* Add documentation for the common bundle resource fields `permissions`, `lifecycle`, and `grants` in the JSON schema, so they surface in editor completions and the docs.
* `bundle run` now prints the modern job run URL (`/jobs/<id>/runs/<id>`) so that non-admin users permitted to view the run are taken to the run instead of the workspace homepage.
* References to a registered model's `registered_model_id` now resolve under the direct engine, matching Terraform behavior ([#5621](#5621)).
* Fix missing field descriptions in the bundle JSON schema for fields whose upstream API docs arrived after the field was first annotated (e.g. `vector_search_endpoints.*.target_qps`); stale placeholder markers no longer hide them ([#5588](#5588)).
* Fix `bundle deploy --plan` dropping a `postgres_role`'s `role_id`, which caused the role to be recreated on the next deploy ([#5672](#5672)).
* direct: Fix spurious cluster recreate when `apply_policy_default_values: true` is set ([#5693](#5693)).
* direct: New 'deployment migrate' implementation that parses terraform state instead of fetching resources state from the backend ([#5399](#5399)).
### Dependency updates
* Bump `github.com/databricks/databricks-sdk-go` from v0.141.0 to v0.147.0 ([#5636](#5636)).
* Bump Terraform provider from v1.117.0 to v1.118.0 ([#5637](#5637)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@denik@eng-dev-ecosystem-bot@andrewnester@shreyas-goenka