Skip to content

libs/dyn/convert: suppress "unknown field" warnings for anchor containers - #5975

Merged
pietern merged 8 commits into
mainfrom
suppress-list-anchor-warnings
Jul 21, 2026
Merged

libs/dyn/convert: suppress "unknown field" warnings for anchor containers#5975
pietern merged 8 commits into
mainfrom
suppress-list-anchor-warnings

Conversation

@pietern

Copy link
Copy Markdown
Contributor

Bundle validation already suppresses "unknown field" warnings for standalone YAML anchors, but still warns when anchors are grouped inside a list or map, the common pattern of collecting reusable blocks under an x-* key:

x-anchors:
- &a { name: foo }- &b { name: bar }

This adds a recursive isAnchorContainer check so a non-empty sequence or map whose elements are all anchors (or all anchor containers) is treated the same as a standalone anchor and does not trigger the warning. Empty containers and containers with any non-anchor element still warn as before. The unknown field is dropped from the normalized output regardless; only the spurious warning is suppressed.

This pull request and its description were written by Isaac.

…ners
Bundle validation already suppresses "unknown field" warnings for
standalone YAML anchors, but still warns when anchors are grouped inside
a list or map, the common pattern of collecting reusable blocks under an
x-* key:
x-anchors:
- &a { name: foo }
- &b { name: bar }
This adds a recursive isAnchorContainer check so a non-empty sequence or
map whose elements are all anchors (or all anchor containers) is treated
the same as a standalone anchor and does not trigger the warning. Empty
containers and containers with any non-anchor element still warn as
before. The unknown field is dropped from the normalized output
regardless; only the spurious warning is suppressed.
Co-authored-by: Isaac
The suppression does not depend on the x-* prefix, so use plain keys.
Co-authored-by: Isaac
Co-authored-by: Isaac
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 286272a

Run: 29731030706

Env💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux4422711224:34
💚​aws windows4422911206:42
💚​aws-ucws linux4431410395:03
💚​aws-ucws windows4431610378:00
💚​azure linux4422711214:15
💚​azure windows4422911196:49
💚​azure-ucws linux4431610365:21
💚​azure-ucws windows4431810347:24
💚​gcp linux4422611234:00
💚​gcp windows4422811219:05
8 interesting tests: 4 RECOVERED, 4 SKIP
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
Top 10 slowest tests (at least 2 minutes):
durationenvtestname
8:40gcp windowsTestAccept
6:44aws-ucws windowsTestAccept
6:23azure windowsTestAccept
6:18aws windowsTestAccept
6:16azure-ucws windowsTestAccept
2:58azure linuxTestAccept
2:56gcp linuxTestAccept
2:53azure-ucws linuxTestAccept
2:52aws linuxTestAccept
2:51aws-ucws linuxTestAccept

Comment threadlibs/dyn/convert/normalize.go
The acceptance test only exercised the no-warning path. Extend it to also
cover a container nested inside a container and a mixed container that holds a
non-anchor element (which is reported as an unknown field), documenting the
rationale for not suppressing the mixed case.
Co-authored-by: Isaac
Remove the parenthetical about x-* keys from the isAnchorContainer doc
comment and rename the unit test's unknown field from x-thing to thing, so
neither uses the x- prefix.
Co-authored-by: Isaac
@pietern
pietern enabled auto-merge July 21, 2026 14:20
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 1365452

Run: 29838319195

Env🟨​KNOWN💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws-cli-is linux4431410454:17
💚​aws-cli-is windows4431610436:02
💚​azure-cli-is linux4431410444:32
💚​azure-cli-is windows4431610426:11
🟨​gcp-cli-is linux31431310467:59
🟨​gcp-cli-is windows31431510449:31
8 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED
Test Nameaws-cli-is linuxaws-cli-is windowsazure-cli-is linuxazure-cli-is windowsgcp-cli-is linuxgcp-cli-is windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R🟨​K🟨​K
Top 3 slowest tests (at least 2 minutes):
durationenvtestname
5:09azure-cli-is windowsTestAccept
4:44gcp-cli-is windowsTestAccept
4:05aws-cli-is windowsTestAccept

@pietern
pietern added this pull request to the merge queueJul 21, 2026
Merged via the queue into main with commit 195c987Jul 21, 2026
28 checks passed
@pietern
pietern deleted the suppress-list-anchor-warnings branch July 21, 2026 14:55
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 195c987

Run: 29841495957

Env❌​FAIL🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
❌​aws-cli-is linux21252983868120:09
❌​aws-cli-is windows21462913886217:48
❌​azure-cli-is linux102442868904164:02
❌​azure-cli-is windows17332807922112:28
❌​gcp-cli-is linux55712858910157:23
❌​gcp-cli-is windows74512801928166:37
33 interesting tests: 21 FAIL, 6 KNOWN, 4 flaky, 2 SKIP
Test Nameaws-cli-is linuxaws-cli-is windowsazure-cli-is linuxazure-cli-is windowsgcp-cli-is linuxgcp-cli-is windows
🟨​TestAccept🟨​K💚​R🟨​K🟨​K🟨​K🔄​f
❌​TestAccept/bundle/deploy/spark-jar-task✅​p❌​F❌​F❌​F✅​p🔄​f
❌​TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform✅​p❌​F❌​F❌​F✅​p🔄​f
🔄​TestAccept/bundle/invariant/continue_293✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume.yml.tmpl✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/invariant/migrate✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=pipeline.yml.tmpl✅​p🔄​f✅​p✅​p✅​p✅​p
🟨​TestAccept/bundle/invariant/no_drift💚​R💚​R🟨​K🟨​K🟨​K🟨​K
❌​TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=✅​p🔄​f❌​F❌​F❌​F✅​p
❌​TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1✅​p🔄​f🔄​f❌​F❌​F❌​F
❌​TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=✅​p🔄​f❌​F❌​F❌​F✅​p
❌​TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1✅​p🔄​f🔄​f❌​F🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/deploy/data_security_mode❌​F🔄​f🔄​f❌​F✅​p❌​F
❌​TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct❌​F🔄​f🔄​f❌​F✅​p❌​F
❌​TestAccept/bundle/resources/clusters/deploy/simple✅​p🔄​f✅​p❌​F❌​F❌​F
❌​TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct✅​p🔄​f✅​p❌​F❌​F❌​F
❌​TestAccept/bundle/resources/clusters/deploy/update-after-create🔄​f✅​p✅​p❌​F🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct🔄​f✅​p✅​p❌​F🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/lifecycle-started✅​p🔄​f❌​F✅​p🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/lifecycle-started-toggle✅​p🔄​f❌​F✅​p✅​p❌​F
❌​TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct✅​p🔄​f❌​F✅​p✅​p❌​F
❌​TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct✅​p🔄​f❌​F✅​p🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/resize-terminated-fallback✅​p✅​p❌​F❌​F🔄​f✅​p
❌​TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p❌​F❌​F🔄​f✅​p
❌​TestAccept/bundle/resources/dashboards/change-name✅​p✅​p✅​p❌​F✅​p✅​p
❌​TestAccept/bundle/resources/dashboards/change-name/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p❌​F✅​p✅​p
🟨​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name💚​R💚​R💚​R🟨​K💚​R💚​R
❌​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p❌​F✅​p✅​p
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R🟨​K🟨​K
Top 50 slowest tests (at least 2 minutes):
durationenvtestname
15:09aws-cli-is windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:54aws-cli-is windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
14:54gcp-cli-is linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:46aws-cli-is linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
13:00aws-cli-is linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:21gcp-cli-is windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:10gcp-cli-is windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:06gcp-cli-is linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:31gcp-cli-is windowsTestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
10:35gcp-cli-is windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
10:15azure-cli-is windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
9:52gcp-cli-is windowsTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
9:52gcp-cli-is windowsTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
9:21aws-cli-is linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
9:12aws-cli-is windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:56gcp-cli-is windowsTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
8:41azure-cli-is windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:32aws-cli-is linuxTestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
8:23azure-cli-is linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
8:10aws-cli-is linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:06gcp-cli-is linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
8:02azure-cli-is linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
7:57azure-cli-is windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:56azure-cli-is linuxTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:47gcp-cli-is linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
7:43aws-cli-is windowsTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:33gcp-cli-is linuxTestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct
7:24azure-cli-is linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:21aws-cli-is windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:14azure-cli-is linuxTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:08aws-cli-is linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
7:04aws-cli-is windowsTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:52aws-cli-is linuxTestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:33gcp-cli-is windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
6:22aws-cli-is windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:34gcp-cli-is linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
5:26gcp-cli-is linuxTestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
5:22gcp-cli-is windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:09gcp-cli-is linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
5:02aws-cli-is linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
5:01aws-cli-is linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:48gcp-cli-is linuxTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:41gcp-cli-is linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:36gcp-cli-is linuxTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
4:35gcp-cli-is windowsTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
4:29gcp-cli-is windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:20gcp-cli-is windowsTestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.10
4:16azure-cli-is windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:14gcp-cli-is linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:13azure-cli-is linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct

deco-sdk-taggingBot added a commit that referenced this pull request Jul 22, 2026
## Release v1.9.0
### CLI
* `databricks auth profiles` no longer stalls on an unreachable workspace and instead fails validation after 5 seconds per host ([#5928](#5928)).
* Fixed `databricks fs rm -r` failing on UC Volumes backed by GCS when a directory becomes empty during recursive deletion ([#5958](#5958)).
* You can now ask questions about your data directly from the CLI with `databricks genie ask "..."`. Genie answers natural-language questions ("what were total sales last month?", "which tables are in the sales catalog?"), runs the query inside Databricks, and renders the answer in the terminal. This promotes the former `databricks experimental genie ask` command; the experimental alias still works but is deprecated and will be removed in a future release ([#6010](#6010)).
### Bundles
* `bundle validate` now reports a clear error when a `sql_warehouse` is missing a `name` (including whitespace-only names), and a warning when a grant is missing a `principal` ([#5818](#5818)).
* Bundle templates now scaffold an `AGENTS.md` that points coding agents at Databricks AI Tools, alongside a minimal `CLAUDE.md` that includes it via `@AGENTS.md` ([#5996](#5996)).
* `bundle generate job` can now download workspace files referenced by `spark_python_task`, rewriting them to a relative path like it already does for notebooks. This is opt-in via the `--download-spark-python-files` flag ([#5799](#5799)).
* Simplified the `default-minimal` bundle template and added an alias `databricks bundle init empty` ([#5899](#5899)).
* Add support for the `instance_pools` resource type in Declarative Automation Bundles. Instance pools are only supported in direct deployment mode.
* Do not emit "unknown field" warnings for YAML anchors grouped in a list or map, matching the existing suppression for standalone anchors ([#5975](#5975)).
* Provide an actionable error message if databricks.yml is missing or DATABRICKS_BUNDLE_ROOT is invalid ([#5953](#5953)).
### Dependency Updates
* Bump `github.com/databricks/databricks-sdk-go` from v0.154.0 to v0.160.0 ([#5982](#5982)).
* Bump Terraform provider from v1.121.0 to v1.122.0 ([#5977](#5977)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pietern@eng-dev-ecosystem-bot@andrewnester@janniklasrose