Read dashboard definition from `file_path` via filer interface by anton-107 · Pull Request #2738 · databricks/cli

anton-107 · 2025-04-17T11:55:31Z

Changes

When dashboard resource contains a file_path field, that file is read and its contents are set to the serialized_dashboard field. This is done in a newly introduced ConfigureDashboardSerializedDashboard mutator
Old behavior of wrapping file_path into terraform's file directive is removed from dashboardConverter (unit tests for this are also removed)
ConfigureWSFS mutator call is moved to be executed before PythonMutator so that the latter has access to the Workspace File System client

Why

This change allows bundle deploy to correctly deploy dashboard resource when running in Databricks Workspace / Runtime. Deployment does not work currently, because reading a dashboard file via file system interface throws an error in the Workspace.

Tests

Added a new dashboard deployment acceptance test
Performed a manual test to deploy a dashboard from a workspace

shreyas-goenka

Thanks! Could you also include in the PR description what does not work today in the workspace? Also please take a look at converting the mutator to read from the typed tree. That'll simplify the code quite a bit.

shreyas-goenka

Approving to unblock. Just one question about why we needed to change the file translation.

…ansformation pipeline, closer to terraform write

…tyle

anton-107 · 2025-04-24T08:28:33Z

Approving to unblock. Just one question about why we needed to change the file translation.

yes. this is needed for the filer to read the file correctly

…malizeMutators

denik

Did you manually test it in the workspace? (You can cross-build and upload binary there)

pietern

Can you confirm this change is a no-op for dashboards deployed with an earlier version of the CLI? Everything else LGTM.

anton-107 · 2025-04-29T12:55:45Z

Did you manually test it in the workspace? (You can cross-build and upload binary there)

Yes, tested for new dashboards in workspace - deployment works with this change in

denik · 2025-04-29T12:57:50Z

Did you manually test it in the workspace? (You can cross-build and upload binary there)
Yes, tested for new dashboards in workspace - deployment works with this change in

OK, thanks. I asked because I did not see it not mentioned in Tests section, but I see it's there now.

anton-107 · 2025-04-29T13:15:34Z

Can you confirm this change is a no-op for dashboards deployed with an earlier version of the CLI? Everything else LGTM.

Wrote this test:

created a new bundle containing a simple dashboard
deployed it with a CLI version that does not have this change (latest main
deployed it with a CLI version containing this change

the second run reported Deployment as complete and there were no signs of any changes that were required

## Changes We started inlining the dashboard definition in #2738 by reading `file_path` and setting it in `serialized_dashboard`. This broke the `bundle generate` CUJ for updating an already imported dashboard when working with DABs. This PR fixes that by keeping the `file_path` around in the configuration tree as metadata and only dropping it when we generate the TF configuration. This PR also adds regression tests to ensure this keeps working in the future. ## Why Fixes#2998 (comment). ## Tests Acceptance tests.

…#3006) ## Why  #2738 changed reading the dashboard contents before the dashboard path was normalized. This change fixes that defect, and also fixes use cases of reading dashboards contents inside a bundle, which defines sync root that is different from the bundle root ## Tests  Existing and new acceptance tests

## Changes  1. When dashboard resource contains a `file_path` field, that file is read and its contents are set to the `serialized_dashboard` field. This is done in a newly introduced `ConfigureDashboardSerializedDashboard` mutator 2. Old behavior of wrapping file_path into terraform's `file` directive is removed from `dashboardConverter` (unit tests for this are also removed) 3. `ConfigureWSFS` mutator call is moved to be executed before `PythonMutator` so that the latter has access to the Workspace File System client ## Why  This change allows `bundle deploy` to correctly deploy dashboard resource when running in Databricks Workspace / Runtime. Deployment does not work currently, because reading a dashboard file via file system interface throws an error in the Workspace. ## Tests  - Added a new dashboard deployment acceptance test - Performed a manual test to deploy a dashboard from a workspace

## Changes We started inlining the dashboard definition in #2738 by reading `file_path` and setting it in `serialized_dashboard`. This broke the `bundle generate` CUJ for updating an already imported dashboard when working with DABs. This PR fixes that by keeping the `file_path` around in the configuration tree as metadata and only dropping it when we generate the TF configuration. This PR also adds regression tests to ensure this keeps working in the future. ## Why Fixes#2998 (comment). ## Tests Acceptance tests.

…#3006) ## Why  #2738 changed reading the dashboard contents before the dashboard path was normalized. This change fixes that defect, and also fixes use cases of reading dashboards contents inside a bundle, which defines sync root that is different from the bundle root ## Tests  Existing and new acceptance tests

anton-107 temporarily deployed to test-trigger-is April 17, 2025 11:55 — with GitHub Actions Inactive

anton-107 temporarily deployed to test-trigger-is April 17, 2025 12:14 — with GitHub Actions Inactive

anton-107 marked this pull request as ready for review April 17, 2025 12:51

anton-107 requested review from andrewnester, denik, pietern and shreyas-goenka as code owners April 17, 2025 12:51

shreyas-goenka reviewed Apr 21, 2025
View reviewed changes

anton-107 force-pushed the anton-107/configure-serialized-dashboard branch from c3573b7 to bcba28dCompare April 22, 2025 12:58

anton-107 temporarily deployed to test-trigger-is April 22, 2025 12:58 — with GitHub Actions Inactive

anton-107 temporarily deployed to test-trigger-is April 22, 2025 15:36 — with GitHub Actions Inactive

anton-107 force-pushed the anton-107/configure-serialized-dashboard branch from 28533c6 to d40eabcCompare April 23, 2025 11:12

anton-107 temporarily deployed to test-trigger-is April 23, 2025 11:12 — with GitHub Actions Inactive

anton-107 temporarily deployed to test-trigger-is April 23, 2025 11:55 — with GitHub Actions Inactive

anton-107 requested a review from shreyas-goenka April 23, 2025 12:36

shreyas-goenka approved these changes Apr 23, 2025
View reviewed changes

Comment threadbundle/config/mutator/paths/dashboard_paths_visitor.go

pietern requested changes Apr 24, 2025
View reviewed changes

Comment threadbundle/phases/destroy.go Outdated
Comment threadbundle/phases/initialize.go
Comment threadbundle/config/mutator/paths/dashboard_paths_visitor.go

pietern changed the title ~~Read dashboard definition from file_path via filer interface~~Read dashboard definition from file_path via filer interfaceApr 24, 2025

anton-107 added 6 commits April 24, 2025 10:24

Read dashboard definition from file_path via filer interface
399a8a6

fix acceptance test
c286774

style fix: avoid indentation by returning early when ok is false
efd5faa

refactor: move ConfigureDashboardSerializedDashboard call down the tr…
2e760ee
…ansformation pipeline, closer to terraform write

revert the ConfigureDashboardSerializedDashboard mutator to dynamic s…
f6a27a7
…tyle

small fix: fix test fixture and add a clarifying comment to the mutator
8a8c084

remove unused paths translation mode TranslateModeLocalAbsoluteFile
e7e0b45

anton-107 force-pushed the anton-107/configure-serialized-dashboard branch from abdf67b to e7e0b45Compare April 24, 2025 08:37

anton-107 temporarily deployed to test-trigger-is April 24, 2025 08:37 — with GitHub Actions Inactive

move ConfigureDashboardSerializedDashboard to be called from applyNor…
c3f2bda
…malizeMutators

anton-107 temporarily deployed to test-trigger-is April 24, 2025 12:40 — with GitHub Actions Inactive

anton-107 requested a review from pietern April 24, 2025 14:08

denik reviewed Apr 29, 2025
View reviewed changes

Comment threadbundle/config/mutator/resourcemutator/configure_dashboards_serialized_dashboard.go
Comment threadbundle/config/mutator/paths/dashboard_paths_visitor.go

pietern approved these changes Apr 29, 2025
View reviewed changes

anton-107 added this pull request to the merge queue Apr 29, 2025

Merged via the queue into main with commit 59a4da6Apr 29, 2025

anton-107 deleted the anton-107/configure-serialized-dashboard branch April 29, 2025 13:30

liamperritt mentioned this pull request May 5, 2025
DAB deployments containing AI/BI dashboard resources fail when using the Databricks Web Terminal #2750
Closed

shreyas-goenka mentioned this pull request Jun 9, 2025
Fix dashboard generation for already imported dashboard #3016
Merged

anton-107 mentioned this pull request Jun 10, 2025
Translate dashboards' paths before their serialized contents are read #3006
Merged

Uh oh!

Conversation

anton-107 commented Apr 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

shreyas-goenka left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shreyas-goenka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anton-107 commented Apr 24, 2025

Uh oh!

denik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pietern left a comment

Choose a reason for hiding this comment

Uh oh!

anton-107 commented Apr 29, 2025

Uh oh!

denik commented Apr 29, 2025

Uh oh!

anton-107 commented Apr 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

anton-107 commented Apr 17, 2025•
edited
Loading

shreyas-goenka left a comment •
edited
Loading