Encourage the use of root_path in production to ensure single deployment by lennartkats-db · Pull Request #1712 · databricks/cli

lennartkats-db · 2024-08-22T20:12:38Z

Changes

This updates mode: production to allow root_path to indicate uniqueness. Historically, we required run_as for this, which isn't actually very effective for that purpose. run_as also had the problem that it doesn't work for pipelines.

This is a cherry-pick from #1387

Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>

…encourage-root-path

…-path

lennartkats-db · 2024-12-02T07:52:33Z

@pietern @andrewnester could you take another look at this PR? The remaining thread should be resolved: #1712 (comment)

…-path

pietern · 2025-01-02T15:38:32Z

+ // report an error for them.
+ return diag.Warningf("target with 'mode: production' should " + advice)
+ }
+ return diag.Errorf("target with 'mode: production' must " + advice)


For follow-up: the advice should be a multi-line string in the Detail field to render nicely. As is, it is shown as a single (wrapped) line. The warning case will light up in all cases where mode: production is used and the root path is not specified.
Also, this doc page should be updated https://docs.databricks.com/en/dev-tools/bundles/deployment-modes.html#production-mode as well and linked from the warning to explain why the warning is shown while users haven't changed anything.

github-actions · 2025-01-13T10:03:07Z

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

PR number: 1712
Commit SHA: 9098320c63350f44d07c86da3c071331c2556bf3

Checks will be approved automatically on success.

New feature announcement. You can now manage Databricks Apps using DABs by defining an `app` resource in your bundle configuration. For more information see Databricks documentation https://docs.databricks.com/en/dev-tools/bundles/resources.html#app CLI: * Filter out system clusters in cluster picker ([#2131](#2131)). * Process all the fields in top level request object even if it contains request body ([#2155](#2155)). Bundles: * Added support for Databricks Apps in DABs ([#1928](#1928)). * Allow artifact path to be located outside the sync root ([#2128](#2128)). * Retry app deployment if there is an active deployment in progress ([#2153](#2153)). * Resolve variables in a loop ([#2164](#2164)). * Improve resolution of complex variables within complex variables ([#2157](#2157)). * Added output message to warn about slower deployments with apps ([#2161](#2161)). * Patch references to UC schemas to capture dependencies automatically ([#1989](#1989)). * Format default-python template ([#2110](#2110)). * Encourage the use of root_path in production to ensure single deployment ([#1712](#1712)). * Log warnings to stderr for "bundle validate -o json" ([#2109](#2109)). Internal: * Move merge fix-ups after variable resolution ([#2125](#2125)). * Enable linter 'unconvert' and fix the issues found ([#2136](#2136)). * Coverage for acceptance tests ([#2123](#2123)). * Add acceptance tests for builtin templates ([#2135](#2135)). * Add a unique schema for recreate pipeline test ([#2159](#2159)). * Migrate resolution tests to acceptance tests ([#2143](#2143)). * Update runner for the publish-winget job ([#2105](#2105)). * Add a test for complex variable resolution with 3 levels ([#2163](#2163)). API Changes: * Changed `databricks account federation-policy update` command with new required argument order. * Changed `databricks account service-principal-federation-policy update` command with new required argument order. OpenAPI commit 779817ed8d63031f5ea761fbd25ee84f38feec0d (2025-01-08) Dependency updates: * Upgrade TF provider to 1.63.0 ([#2162](#2162)). * Bump golangci-lint version to v1.63.4 from v1.63.1 ([#2114](#2114)). * Bump astral-sh/setup-uv from 4 to 5 ([#2116](#2116)). * Bump golang.org/x/oauth2 from 0.24.0 to 0.25.0 ([#2080](#2080)). * Bump github.com/hashicorp/hc-install from 0.9.0 to 0.9.1 ([#2079](#2079)). * Bump golang.org/x/term from 0.27.0 to 0.28.0 ([#2078](#2078)). * Bump github.com/databricks/databricks-sdk-go from 0.54.0 to 0.55.0 ([#2126](#2126)).

### New feature announcement #### Databricks Apps support You can now manage Databricks Apps using DABs by defining an `app` resource in your bundle configuration. For more information see Databricks documentation https://docs.databricks.com/en/dev-tools/bundles/resources.html#app #### Referencing complex variables in complex variables You can now reference complex variables within other complex variables. For more details see #2157 CLI: * Filter out system clusters in cluster picker ([#2131](#2131)). * Add command line flags for fields that are not in the API request body ([#2155](#2155)). Bundles: * Added support for Databricks Apps in DABs ([#1928](#1928)). * Allow artifact path to be located outside the sync root ([#2128](#2128)). * Retry app deployment if there is an active deployment in progress ([#2153](#2153)). * Resolve variables in a loop ([#2164](#2164)). * Improve resolution of complex variables within complex variables ([#2157](#2157)). * Added output message to warn about slower deployments with apps ([#2161](#2161)). * Patch references to UC schemas to capture dependencies automatically ([#1989](#1989)). * Format default-python template ([#2110](#2110)). * Encourage the use of root_path in production to ensure single deployment ([#1712](#1712)). * Log warnings to stderr for "bundle validate -o json" ([#2109](#2109)). API Changes: * Changed `databricks account federation-policy update` command with new required argument order. * Changed `databricks account service-principal-federation-policy update` command with new required argument order. OpenAPI commit 779817ed8d63031f5ea761fbd25ee84f38feec0d (2025-01-08) Dependency updates: * Upgrade TF provider to 1.63.0 ([#2162](#2162)). * Bump golangci-lint version to v1.63.4 from v1.63.1 ([#2114](#2114)). * Bump astral-sh/setup-uv from 4 to 5 ([#2116](#2116)). * Bump golang.org/x/oauth2 from 0.24.0 to 0.25.0 ([#2080](#2080)). * Bump github.com/hashicorp/hc-install from 0.9.0 to 0.9.1 ([#2079](#2079)). * Bump golang.org/x/term from 0.27.0 to 0.28.0 ([#2078](#2078)). * Bump github.com/databricks/databricks-sdk-go from 0.54.0 to 0.55.0 ([#2126](#2126)). --------- Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>

## Changes This refines the `mode: production` diagnostic output now that the `Detail` property is rendered as output. This is a follow-up to #1712.

## Changes This removes the `run-as` property from the default templates. It's a useful property but it still only works for jobs and it makes the default databricks.yml a bit longer. It seems like users can just learn about it from the docs and/or vary their deployment identity. Depends on #1712.

…ent (#1712) ## Changes This updates `mode: production` to allow `root_path` to indicate uniqueness. Historically, we required `run_as` for this, which isn't actually very effective for that purpose. `run_as` also had the problem that it doesn't work for pipelines. This is a cherry-pick from #1387 --------- Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>

### New feature announcement #### Databricks Apps support You can now manage Databricks Apps using DABs by defining an `app` resource in your bundle configuration. For more information see Databricks documentation https://docs.databricks.com/en/dev-tools/bundles/resources.html#app #### Referencing complex variables in complex variables You can now reference complex variables within other complex variables. For more details see #2157 CLI: * Filter out system clusters in cluster picker ([#2131](#2131)). * Add command line flags for fields that are not in the API request body ([#2155](#2155)). Bundles: * Added support for Databricks Apps in DABs ([#1928](#1928)). * Allow artifact path to be located outside the sync root ([#2128](#2128)). * Retry app deployment if there is an active deployment in progress ([#2153](#2153)). * Resolve variables in a loop ([#2164](#2164)). * Improve resolution of complex variables within complex variables ([#2157](#2157)). * Added output message to warn about slower deployments with apps ([#2161](#2161)). * Patch references to UC schemas to capture dependencies automatically ([#1989](#1989)). * Format default-python template ([#2110](#2110)). * Encourage the use of root_path in production to ensure single deployment ([#1712](#1712)). * Log warnings to stderr for "bundle validate -o json" ([#2109](#2109)). API Changes: * Changed `databricks account federation-policy update` command with new required argument order. * Changed `databricks account service-principal-federation-policy update` command with new required argument order. OpenAPI commit 779817ed8d63031f5ea761fbd25ee84f38feec0d (2025-01-08) Dependency updates: * Upgrade TF provider to 1.63.0 ([#2162](#2162)). * Bump golangci-lint version to v1.63.4 from v1.63.1 ([#2114](#2114)). * Bump astral-sh/setup-uv from 4 to 5 ([#2116](#2116)). * Bump golang.org/x/oauth2 from 0.24.0 to 0.25.0 ([#2080](#2080)). * Bump github.com/hashicorp/hc-install from 0.9.0 to 0.9.1 ([#2079](#2079)). * Bump golang.org/x/term from 0.27.0 to 0.28.0 ([#2078](#2078)). * Bump github.com/databricks/databricks-sdk-go from 0.54.0 to 0.55.0 ([#2126](#2126)). --------- Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>

## Changes This removes the `run-as` property from the default templates. It's a useful property but it still only works for jobs and it makes the default databricks.yml a bit longer. It seems like users can just learn about it from the docs and/or vary their deployment identity. Depends on #1712.

## Changes This refines the `mode: production` diagnostic output now that the `Detail` property is rendered as output. This is a follow-up to #1712.

Encourage the use of root_path in production to ensure single deployment
2dad625

lennartkats-db requested a review from pietern August 28, 2024 07:43

pietern reviewed Aug 28, 2024
View reviewed changes

lennartkats-dband others added 2 commits August 29, 2024 16:49

Update bundle/config/mutator/process_target_mode.go
595beed
Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>

Add nil check
821239e

lennartkats-db requested a review from pietern September 2, 2024 12:01

pietern reviewed Sep 6, 2024
View reviewed changes

lennartkats-db added 2 commits September 9, 2024 09:20

Extend message
dc63c7c

Merge remote-tracking branch 'origin/cp-encourage-root-path' into cp-…
5436123
…encourage-root-path

pietern requested a review from andrewnester September 13, 2024 10:33

lennartkats-db requested a review from pietern October 5, 2024 03:59

Merge remote-tracking branch 'databricks/main' into cp-encourage-root…
3f45f71
…-path

lennartkats-db enabled auto-merge October 12, 2024 09:56

lennartkats-db force-pushed the cp-encourage-root-path branch 2 times, most recently from 742210c to 3973c50Compare October 12, 2024 10:01

Merge CleanupTarget back into SelectTarget
6ea5306

lennartkats-db force-pushed the cp-encourage-root-path branch from 3973c50 to 6ea5306Compare October 17, 2024 17:31

pietern reviewed Dec 2, 2024
View reviewed changes

Comment threadbundle/config/bundle.go Outdated
Comment threadbundle/config/mutator/process_target_mode.go Outdated

lennartkats-db added 2 commits December 8, 2024 20:45

Merge remote-tracking branch 'databricks/main' into cp-encourage-root…
c5fa11c
…-path

Move config.Bundle.TargetConfig to bundle.Bundle.Target
293d780

lennartkats-db temporarily deployed to test-trigger-is December 9, 2024 08:38 — with GitHub Actions Inactive

lennartkats-db disabled auto-merge December 9, 2024 08:38

lennartkats-db requested a review from pietern December 9, 2024 08:40

lennartkats-db mentioned this pull request Dec 22, 2024
Remove run_as from the built-in templates #2044
Merged

pietern reviewed Jan 2, 2025
View reviewed changes

pietern approved these changes Jan 2, 2025
View reviewed changes

Use a recommendation instead of a warning
3e01aec

lennartkats-db enabled auto-merge January 10, 2025 12:42

lennartkats-db disabled auto-merge January 10, 2025 12:43

lennartkats-db enabled auto-merge January 10, 2025 12:44

lennartkats-db disabled auto-merge January 10, 2025 12:44

lennartkats-db enabled auto-merge January 10, 2025 13:13

lennartkats-db removed the request for review from andrewnester January 10, 2025 17:45

lennartkats-db added this pull request to the merge queue Jan 11, 2025

github-merge-queueBot removed this pull request from the merge queue due to failed status checks Jan 11, 2025

lennartkats-db added 2 commits January 13, 2025 10:56

Merge remote-tracking branch 'origin/main' into cp-encourage-root-path
3b63fb2

Fix linter messages
9098320

lennartkats-db requested review from andrewnester, denik and shreyas-goenka as code owners January 13, 2025 10:02

lennartkats-db added this pull request to the merge queue Jan 13, 2025

Merged via the queue into databricks:main with commit 3e40a0cJan 13, 2025

lennartkats-db deleted the cp-encourage-root-path branch January 13, 2025 12:24

andrewnester mentioned this pull request Jan 16, 2025
[Release] Release v0.239.0 #2167
Merged

lennartkats-db mentioned this pull request Jan 27, 2025
Refine mode: production diagnostic output #2236
Merged

github-merge-queueBot pushed a commit that referenced this pull request Feb 24, 2025
Refinemode: productiondiagnostic output (#2236)
6e18d94
## Changes This refines the `mode: production` diagnostic output now that the `Detail` property is rendered as output. This is a follow-up to #1712.

denik pushed a commit that referenced this pull request May 20, 2026
Refinemode: productiondiagnostic output (#2236)
fe98c6e
## Changes This refines the `mode: production` diagnostic output now that the `Detail` property is rendered as output. This is a follow-up to #1712.

Navigation Menu

Uh oh!

Encourage the use of root_path in production to ensure single deployment - #1712

Encourage the use of root_path in production to ensure single deployment#1712
lennartkats-db merged 12 commits into
databricks:mainfrom
lennartkats-db:cp-encourage-root-path

lennartkats-db commented Aug 22, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lennartkats-db commented Dec 2, 2024

Uh oh!

Uh oh!

Uh oh!

pieternJan 2, 2025

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Jan 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

lennartkats-db commented Aug 22, 2024

Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lennartkats-db commented Dec 2, 2024

Uh oh!

Uh oh!

Uh oh!

pieternJan 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Jan 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants