Uh oh!
There was an error while loading. Please reload this page.
internal: Add support for 'sensitive' JSON tag for bundle config fields - #5896
Merged
Conversation
andrewnester
requested review from
denik, janniklasrose, pietern and shreyas-goenkaJuly 10, 2026 13:08
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 13:37 — with 
GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 13:37 — with GitHub Actions
Inactive
Collaborator
Integration test reportCommit: 49ea5f6
9 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED, 1 flaky
Top 8 slowest tests (at least 2 minutes):
|
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 14:44 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 14:44 — with GitHub Actions
Inactive
andrewnesterhad a problem deploying
to
test-trigger-is
July 10, 2026 14:54 — with GitHub Actions
Failure
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 14:54 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 15:11 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 10, 2026 15:11 — with GitHub Actions
Inactive
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 09:07 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 09:07 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 09:23 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 09:23 — with GitHub Actions
Inactive
pietern
reviewed
Jul 14, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| } | ||
| return si.Sensitive | ||
| } | ||
Contributor
There was a problem hiding this comment.
Please add a few unit tests here in dyn/convert that demonstrates that this works in cases with embedded structs, anonymous structs, etc. Would also be good to include a user of these fields in the same package.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 12:27 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 12:27 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 12:40 — with GitHub Actions
Inactive
andrewnestertemporarily deployed
to
test-trigger-is
July 14, 2026 12:40 — with GitHub Actions
Inactive
pietern
reviewed
Jul 15, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
pietern
approved these changes
Jul 21, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| case dyn.KindString: | ||
| // Preserve sensitive strings: NewValue would strip the secretString wrapper. | ||
| if src.IsSensitive() { | ||
| return src, nil |
Contributor
There was a problem hiding this comment.
This means it is not possible to update the secret value in the typed tree.
If expected, please include this expectation in the comment.
andrewnester
enabled auto-merge
July 22, 2026 12:24
eng-dev-ecosystem-bot
commented
Jul 22, 2026
Collaborator
Integration test reportCommit: 40ee185
37 interesting tests: 27 FAIL, 6 KNOWN, 2 flaky, 2 SKIP
Top 50 slowest tests (at least 2 minutes):
|
janniklasrose added a commit
that referenced
this pull request
Aug 5, 2026
## Changes Revert usage of RedactSensitiveFields ## Why RedactSensitiveFields (added in #5896, merged 2026-07-22) deep-clones the state struct field-by-field via reflection, then marshals the clone. The clone skips unexported fields (libs/structs/structwalk/redact.go:44-46), and duration.Duration stores its whole value in an unexported field while marshalling through a custom MarshalJSON So on main, every duration written to the direct-engine state file is silently corrupted to 0s. ## Tests Golden fiels updated <!-- If your PR needs to be included in the release notes for next release, add a changelog fragment: create .nextchanges/<section>/<name>.md with a one-line description (e.g. .nextchanges/cli/quickstart.md). See .nextchanges/README.md. --> --------- Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
deco-sdk-taggingBot
added a commit
that referenced
this pull request
Aug 6, 2026
## Release v1.11.0 ### CLI * Fixed `databricks repos get/update/delete` failing with `object at path "..." is not a repo` for Git-CLI-enabled folders (currently in preview), which the workspace API reports as directories rather than repos ([#6181](#6181)). * Support `dbfs:/Skills/...` paths in `databricks fs` commands, routed to the Files API. ([#6147](#6147)) ### Bundles * For jobs where `ai_runtime_task.code_source_path` is a relative path to a local directory, the directory is now packaged into a tarball (honoring `.gitignore` and `sync.include`/`sync.exclude`), uploaded during deployment, and `code_source_path` is rewritten to the uploaded workspace path. ([#6110](#6110)) * Added JSON output to `bundle init`. Running `databricks bundle init <template> -o json` now reports the files the template wrote, relative to the output directory. This lets callers that pass `--output-dir` learn where the template materialized instead of assuming the output is a single directory named after the project. The default text output is unchanged. ([#6161](#6161)) * The terraform deployment engine is deprecated and will stop working in a future version of the CLI. Setting `bundle.engine: terraform` now emits a deprecation warning. See https://docs.databricks.com/aws/en/dev-tools/bundles/direct for how to migrate to the direct deployment engine. ([#6099](#6099)) * Fixed the direct deployment engine planning a spurious `create` for an empty `grants: []` list. Terraform records no grants resource for such a list, so `bundle plan` after `bundle deployment migrate` no longer reports an action for it. Emptying a previously deployed list still revokes the grants, after which the node is dropped from the deployment state instead of being reported as unchanged forever. ([#6039](#6039)) * Fixed `bundle generate` downloading notebooks found inside a folder without their file extension. They are now exported like top-level notebooks, so a Python notebook lands as `notebook.py` instead of an extensionless file ([#6144](#6144)). * direct: `webhook_notifications.on_*` destinations on jobs, tasks, and `for_each_task` are now compared as unordered sets. Previously the Jobs API returning these lists in a different order than submitted produced a phantom diff that `bundle plan` and `bundle deploy` could never converge past, reporting `1 to change` on every run ([#6060](#6060)). * Fixed a pipeline with `allow_duplicate_names: true` never converging on the direct engine: the field is only accepted on create/update and is never returned by the pipelines GET API, so every subsequent `bundle plan` reported the pipeline as a perpetual update. ([#6076](#6076)) * direct: A local change to an input-only field (one the API accepts on write but never returns on read, e.g. pipelines' `run_as` or external locations' `skip_validation`) is no longer silently skipped when the new value coincidentally matches the field's fabricated remote value. Previously such a change could hit the `remote_already_set` shortcut and be dropped from the plan. ([#6112](#6112)) * Revert usage of RedactiveSenstiveFields (added in [#5896](#5896), released in 1.10.0) which lead to incorrect behaviour (permanent drift) for duration field in Postgres resources ([#6179](#6179)). * Document postgres resource fields in the json schema ([#6164](#6164), [#6163](#6163)). * direct: Recreating a `vector_search_indexes` resource no longer fails with "Index ... is currently pending deletion" when the backend has not yet released the index name. The create is now retried until the name becomes available. ([#6143](#6143)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.165.0 to v0.166.0. ([#6175](#6175)) * Upgrade Terraform provider to 1.124.0. ([#6174](#6174))
andrewnester added a commit
that referenced
this pull request
Aug 6, 2026
## Changes
Add support for Unity Catalog (UC) secrets as a first-class DAB resource
type under the direct deployment engine.
Notes:
- Secret values are never persisted to deployment state — the remote
plaintext is not returned by the API and is intentionally omitted from
state to avoid leaking secrets
- New validation mutator (ValidateSecretValueIsVariable) that rejects
plain-text value fields at bundle validation time — the value must be a
variable reference (e.g. ${var.my_secret}) so secrets are never
committed to databricks.yml
Fixes#3689
Also requires #5896
## Example
```
bundle:
name: my-bundle
variables:
my_secret_value:
description: "Secret value passed at deploy time"
resources:
secrets:
my_secret:
catalog_name: main
schema_name: default
name: my_api_key
value: ${var.my_secret_value}
comment: "API key for external service"
```
## Tests
Acceptance tests covering create/update/delete, update-value,
direct-only enforcement, and the plain-text validation error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add support for 'sensitive' JSON tag for bundle config fields
Why
This allows us to mark certain fields (for example, secrets.value) as sensitive and let CLI handle masking it in command outputs.
Used in #5861
Tests
Added unit tests. Will be tests end to end with unit tests with UC secrets