Add support for UC secrets by andrewnester · Pull Request #5861 · databricks/cli

andrewnester · 2026-07-08T12:48:12Z

Changes

Add support for Unity Catalog (UC) secrets as a first-class DAB resource type under the direct deployment engine.

Notes:

Secret values are never persisted to deployment state — the remote plaintext is not returned by the API and is intentionally omitted from state to avoid leaking secrets
New validation mutator (ValidateSecretValueIsVariable) that rejects plain-text value fields at bundle validation time — the value must be a variable reference (e.g. ${var.my_secret}) so secrets are never committed to databricks.yml

Also requires #5896

Example

bundle:
name: my-bundle
variables:
my_secret_value:
description: "Secret value passed at deploy time"
resources:
secrets:
my_secret:
catalog_name: main
schema_name: default
name: my_api_key
value: ${var.my_secret_value}
comment: "API key for external service"

Tests

Acceptance tests covering create/update/delete, update-value, direct-only enforcement, and the plain-text validation error

eng-dev-ecosystem-bot · 2026-07-08T13:19:46Z

Integration test report

Commit: bb8e450

Run: 31090074644

	Env	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux	4	4	293	1109	5:22
💚	aws windows	4	4	295	1107	6:31
💚	azure linux	4	4	292	1109	6:02
💚	azure windows	4	4	294	1107	5:05
💚	gcp linux	1	5	293	1109	6:09
💚	gcp windows	1	5	295	1107	5:25

8 interesting tests: 4 RECOVERED, 4 SKIP

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R
💚	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
5:02	aws windows	TestAccept
4:25	gcp windows	TestAccept
3:03	azure windows	TestAccept

shreyas-goenka · 2026-07-13T08:19:36Z

+variables:
+ secret_value:
+ description: "Test secret value"
+ default: "initial-secret-value"


Should we disallow this pattern? Error if we detect this? Secrets should not be stored in plain text in YAML.

janniklasrose · 2026-07-14T15:21:54Z

+// Terraform provider implementation:
+// https://github.com/databricks/terraform-provider-databricks/blob/main/catalog/resource_secret.go


I think we can drop this comment

janniklasrose · 2026-08-05T11:37:21Z

 v.SchemaName = resolveSchema(b, v.CatalogName, v.SchemaName)
 v.CatalogName = resolveCatalog(b, v.CatalogName)
 	}
+	for _, s := range b.Config.Resources.Secrets {


Curious (about capture_uc_dependencies in general): is this just a quality of life improvement if customers have their catalog & schema in DABs but don't use the in-bundle reference?

yes, correct

janniklasrose · 2026-08-05T11:45:48Z

+ return nil, err
+	}
+
+	// SDK does not support include_value in the GetSecretRequest, so we use the API directly.


any reason why? We can (ask to) fix this upstream and switch to the SDK method later

Not sure why, it's a query param though so I suspect there might not be support for such in SDK yet

janniklasrose · 2026-08-05T11:47:50Z

+	state.Value = state.SecretValue
+	response, err := r.client.SecretsUc.CreateSecret(ctx, catalog.CreateSecretRequest{
+ Secret: state.Secret,
+	})
+	// Clear the plaintext so it is not written to the state file.
+	// Fingerprint already captures whether the value changed.
+	state.Value = ""


why are we writing state.SecretValue into state.Value, not using it, then clearing state.Value again? is the read of state.Secret derived from Value? if so a comment would help clarify that side-effect.

ditto below on DoUpdate

why are we writing state.SecretValue into state.Value, not using it,
We use it below when passing state.Secret, state.Value is practically state.Secret.Value

a comment like
// temporarily assign `state.Value` so `state.Secret` has access
would help the (future) reader

## Release v1.12.0 ### CLI * `databricks aitools install` now supports Gemini CLI, installing Databricks agent skills into its skills directory. * `databricks aitools install` now supports Pi, installing Databricks agent skills into its skills directory. * A locally built CLI (`go build`, without release flags) now reports the next release version with a `-dev` prerelease, e.g. `1.12.0-dev+abcdef123456`, instead of `0.0.0-dev+abcdef123456`. The old string sorted below every published release even though a local build is newer than the latest release; the new one sorts above the latest release and below the release it will become, matching what goreleaser already produces for snapshot builds. * Added the `databricks environments setup-local` command, which provisions (or updates) a local Python environment matched to a Databricks compute target. It resolves the target to an environment key, fetches the pinned Python version, databricks-connect version, and dependency constraints published for that key, then provisions a matched `.venv` with uv. ### Bundles * Added a `cascade_on_destroy` field to the pipeline resource to control whether destroying a pipeline also deletes its datasets (MVs, STs, Views). When unset, the server default applies; set `cascade_on_destroy: false` to retain the datasets on destroy. Supported with the direct deployment engine ([#5846](#5846)). * Fix `bundle.deployment.lock.force` being ignored. The `--force-lock` flag's default value overwrote the value configured in `databricks.yml`, so setting the field had no effect and a stale deployment lock could only be overridden with the flag. ([#6188](#6188)) * direct: experimental `job_runs` now sends a CLI-managed idempotency token on every run-now, so an SDK retry after a lost response returns the same run. Configured `idempotency_token` values are rejected. * direct: the experimental `job_runs` resource now waits for the triggered run to finish, so other resources can reference its outcome (e.g. `${resources.job_runs.nightly.state.result_state}`). A run that does not succeed fails the deploy, naming the failed task, and is run again on the next deploy. If a deploy is interrupted while waiting, the next one resumes waiting on the same run. * direct: Fixed model serving `telemetry_config` drift and applied planned telemetry updates. Unsupported endpoint types now fail when telemetry is applied; create may still succeed because it drops the field ([#6106](#6106)). * The `cli_version` field in the direct engine's deployment state (`resources.json`) now records the CLI version that last wrote the state. Previously it kept the version of the CLI that first created the state. * Add support for UC secrets resource ([#5861](#5861)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.166.0 to v0.169.0. * Bump Terraform provider from v1.124.0 to v1.126.0 ([#6250](#6250)).

andrewnester temporarily deployed to test-trigger-is July 8, 2026 12:48 — with GitHub Actions Inactive

andrewnester requested review from denik, janniklasrose, pietern and shreyas-goenka July 8, 2026 12:49

andrewnester temporarily deployed to test-trigger-is July 9, 2026 08:56 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 09:28 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 09:51 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 10:05 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 10:51 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 11:21 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 11:45 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 9, 2026 12:45 — with GitHub Actions Inactive

This was referenced Jul 9, 2026
Add "Secrets" to DAB Resources #3689
Closed
internal: Add support for 'sensitive' JSON tag for bundle config fields #5896
Merged

shreyas-goenka reviewed Jul 13, 2026
View reviewed changes

andrewnester requested a review from shreyas-goenka July 14, 2026 09:51

andrewnester temporarily deployed to test-trigger-is July 14, 2026 09:52 — with GitHub Actions Inactive

andrewnester temporarily deployed to test-trigger-is July 14, 2026 14:57 — with GitHub Actions Inactive

andrewnester added 2 commits July 30, 2026 13:16

next changelog
25a78fd

fixes
77d44ee

denik reviewed Jul 30, 2026
View reviewed changes

Comment threadacceptance/bundle/refschema/out.fields.txt Outdated
Comment threadbundle/direct/dresources/resources.yml Outdated
Comment threadbundle/direct/dresources/secret.go Outdated

andrewnester added 2 commits August 3, 2026 15:50

compare on value
c904d13

diff on value
65899b5

andrewnester requested a review from denik August 3, 2026 13:58

andrewnesterand others added 4 commits August 3, 2026 15:59

remove unused code
a4c3eff

test fixes
8c52c03

refschema + remove not drift
8d772f9

Merge branch 'main' into feat/uc-secrets
ac7171a

janniklasrose reviewed Aug 5, 2026
View reviewed changes

andrewnester requested a review from janniklasrose August 5, 2026 12:18

Merge branch 'main' into feat/uc-secrets
08ad2fc

janniklasrose approved these changes Aug 5, 2026
View reviewed changes

andrewnesterand others added 3 commits August 5, 2026 15:59

Merge branch 'main' into feat/uc-secrets
a4d1b10

update out files
9876d35

update no drift
c3ea248

denik reviewed Aug 5, 2026
View reviewed changes

Comment thread...ptance/bundle/resources/postgres_endpoints/update_autoscaling/out.plan.no_change.direct.json

use the same strcut as state
ce49551

andrewnester requested a review from denik August 5, 2026 15:50

andrewnesterand others added 4 commits August 5, 2026 17:54

fixes
3d98f02

Merge branch 'main' into feat/uc-secrets
970fdcc

fix structs
1235f60

fix invariant
bb8e450

andrewnester disabled auto-merge August 6, 2026 11:01

andrewnester merged commit 7eb1fc2 into mainAug 6, 2026
48 checks passed

andrewnester deleted the feat/uc-secrets branch August 6, 2026 11:02

rugpanov mentioned this pull request Aug 12, 2026
Update Databricks CLI to v1.12.1 databricks/databricks-vscode#2109
Merged

peter-ocelka mentioned this pull request Aug 14, 2026
secrets (uc) - grants & tags #6283
Closed

		// Terraform provider implementation:
		// https://github.com/databricks/terraform-provider-databricks/blob/main/catalog/resource_secret.go

Uh oh!

Conversation

andrewnester commented Jul 8, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example

Tests

Uh oh!

eng-dev-ecosystem-bot commented Jul 8, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andrewnester commented Jul 8, 2026•
edited
Loading

eng-dev-ecosystem-bot commented Jul 8, 2026•
edited
Loading