Skip to content

Add sensitive_fields to resources.yml - #6192

Merged
andrewnester merged 19 commits into
mainfrom
acc/secrets-no-drift
Aug 12, 2026
Merged

Add sensitive_fields to resources.yml#6192
andrewnester merged 19 commits into
mainfrom
acc/secrets-no-drift

Conversation

@andrewnester

@andrewnesterandrewnester commented Aug 6, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Changes

Add sensitive_fields to resources.yml
As a result it allows us to use Value field from SDK struct as is and do a proper drift detected

Why

The effective_value returned from the API after DoRead (with include_value=true) was not being used in the drift comparison, causing a false "update" action on every second plan.

Now we do not unset the fields and redact them instead which makes drift detection reliable.

Tests

Drift test pass

denik
denik reviewed Aug 6, 2026
View reviewed changes
Comment threadbundle/direct/bundle_plan.go Outdated
@andrewnester
andrewnester requested a review from denikAugust 6, 2026 16:01
denik
denik reviewed Aug 7, 2026
View reviewed changes
Comment threadacceptance/bundle/invariant/no_drift/test.toml
Comment threadbundle/direct/dresources/all_test.go Outdated
Comment threadbundle/direct/dresources/secret.go Outdated
Comment threadbundle/direct/bundle_plan.go Outdated
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Aug 7, 2026•
edited
Loading

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: c0794e6

Run: 31596126861

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux5329614128:05
🔄​aws windows143298141010:14
💚​azure linux5329514128:46
💚​azure windows5329714106:30
💚​gcp linux2429614127:38
💚​gcp windows2429814107:44
8 interesting tests: 4 RECOVERED, 3 SKIP, 1 flaky
Test Nameaws linuxaws windowsazure linuxazure windowsgcp linuxgcp windows
🔄​TestAccept💚​R🔄​f💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/invariant/no_drift💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R
Top 11 slowest tests (at least 2 minutes):
durationenvtestname
4:57gcp windowsTestAccept
3:17azure windowsTestAccept
3:03azure windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=1
2:50azure windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=
2:41aws windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=
2:39azure linuxTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=
2:36aws windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=1
2:35gcp windowsTestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=1
2:33aws windowsTestAccept
2:16aws linuxTestAccept/bundle/resources/secrets/basic/DATABRICKS_BUNDLE_ENGINE=direct
2:05gcp windowsTestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret.yml.tmpl/READPLAN=

@andrewnester
andrewnester requested a review from denikAugust 7, 2026 10:07
denik
denik reviewed Aug 10, 2026
View reviewed changes
Comment threadbundle/direct/dresources/secret.go Outdated
Comment threadacceptance/bundle/invariant/no_drift/test.toml
Comment threadbundle/direct/dresources/secret.go Outdated
Comment threadlibs/structs/structdiff/diff.go Outdated
@andrewnesterandrewnester changed the title Fix drift for UC secrets valuesAdd sensitive_fields to resources.ymlAug 10, 2026
denik
denik reviewed Aug 11, 2026
View reviewed changes
Comment threadacceptance/bundle/invariant/delete_idempotent/test.toml Outdated
Comment threadacceptance/bundle/invariant/migrate/test.toml
Comment threadbundle/direct/dresources/redact_config.go Outdated
Comment threadbundle/direct/redact.go Outdated
Comment threadbundle/direct/dresources/redact_config.go
return v, nil
}

const sensitiveRedactedMarker = "[redacted]"

@denikdenikAug 11, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One possible idea: instead of using "[redacted]", use "[redacted_uniquekey]" and keep map of unique key to secret. Then this uniquekey propagates all the way to resource where we can do search and replace post serialization.

denik
denik reviewed Aug 12, 2026
View reviewed changes
Comment threadbundle/direct/dresources/adapter.go Outdated
Comment threadbundle/direct/dresources/redact_config.go Outdated
Comment threadbundle/direct/redact.go Outdated
Comment threadbundle/direct/dresources/serialize_test.go Outdated
denik
denik reviewed Aug 12, 2026
View reviewed changes
Comment threadbundle/direct/apply.go Outdated
Comment threadbundle/direct/redact.go Outdated
Comment threadbundle/direct/dresources/secret.go Outdated
denik
denik approved these changes Aug 12, 2026
View reviewed changes

@denikdenik left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do any tests record bits of resources.json that show that redaction happened here?

@andrewnester
andrewnester added this pull request to the merge queueAug 12, 2026
Merged via the queue into main with commit 0d0f12eAug 12, 2026
25 checks passed
@andrewnester
andrewnester deleted the acc/secrets-no-drift branch August 12, 2026 13:04
@andrewnester

andrewnester commented Aug 12, 2026

Copy link
Copy Markdown
ContributorAuthor

@denik yes, we do it here: https://github.com/databricks/cli/pull/6192/changes#diff-e25f34c90845083428e3fb16df01aaeb6d2b671e96ca64bb190d0a717d426b80R118-R136

denik added a commit that referenced this pull request Aug 13, 2026
@denik
acc: regenerate goldens for tests added on main during rebase
854fe80
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
@denik
acc: regenerate goldens for tests added on main during rebase
0a67817
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
@denik
acc: regenerate goldens for tests added on main during rebase
21790e5
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denikdenikdenik approved these changes
@janniklasrosejanniklasroseAwaiting requested review from janniklasrose

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewnester@eng-dev-ecosystem-bot@denik