Uh oh!
There was an error while loading. Please reload this page.
schema: document the bundle-only fields on postgres resources - #6164
Merged
janniklasrose merged 3 commits intoAug 5, 2026
Conversation
## Problem `postgres_*` resources appear in the bundle JSON schema with their fields but with no descriptions and no launch-stage prefixes, even though `.codegen/cli.json` documents every one of them and marks them `PUBLIC_BETA`. `findRef` binds a config type to a spec schema by checking the type itself and its *direct* anonymous embeds only. That works for resources that embed the SDK type directly (`Job` -> `jobs.JobSettings`), but the postgres resources interpose a config struct so `ForceSendFields` is recorded on the struct that declares each field: PostgresProject -> PostgresProjectConfig -> postgres.ProjectSpec `jsonschema.FromType` flattens embedded structs at any depth, so the Spec's fields land in the schema, but `findRef` stopped one level short and never found the schema documenting them. ## Solution Traverse embedded structs breadth first, mirroring `getStructFields` in `libs/jsonschema/from_type.go`, so the shallowest SDK type present in the spec still wins. The lookup itself moves to `lookupSDKType`. ## Effect Regenerating drops 40 now-stale `PLACEHOLDER` markers from `annotations.yml` (`dropShadowingPlaceholders` prunes a marker once upstream documents the field) and adds the inherited descriptions to `jsonschema.json`. Exactly the 7 postgres resource definitions change; no other definition in the schema is touched and none are added or removed. Two fields that were silently exposed are now labelled: `accelerated_sync` and `extra_columns` on `postgres_synced_tables` pick up `[Private Preview]`, `x-databricks-launch-stage`, and `doNotSuggest`. The bundle-only fields (`parent`, the `*_id`s, `replace_existing`, `purge_on_delete`) stay `PLACEHOLDER`: the spec documents them on sibling schemas (`postgres.Create*Request`, and the `postgres.Branch` / `Endpoint` / `Database` / `Role` envelopes) that no bundle type embeds. They need hand-authored text, which is a separate change. ## Tests `TestFindRefNestedEmbeddedSDKType` and `TestExtractAnnotationsNestedEmbeddedSDKType` cover a spec embedded below the first level, using the same shape as the postgres resources.
The postgres_* resources wrap SDK specs with fields that bundles own: the *_id components of the hierarchical resource name, parent, replace_existing, and purge_on_delete. None of these exist on the spec the annotations are inherited from, so they generated as PLACEHOLDER. Fill them in from the upstream Create*Request docs where the semantics match, and hand-write the rest. The takeover and purge fields describe bundle behaviour rather than a single API call, and postgres_branches.parent points at source_branch instead of the upstream status.source_branch, which bundles flatten away.
eng-dev-ecosystem-bot
commented
Aug 4, 2026
Collaborator
Integration test reportCommit: 3972261
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 6 slowest tests (at least 2 minutes):
|
denik
approved these changes
Aug 5, 2026
Base automatically changed from
janniklasrose/postgres-schema-annotations-propagate to
mainAugust 5, 2026 12:40
janniklasrose
enabled auto-merge
August 5, 2026 12:42
janniklasrose
disabled auto-merge
August 5, 2026 13:38
janniklasrose
commented
Aug 5, 2026
ContributorAuthor
skip windows integ tests due to shortage of runners. linux ones are green |
Uh oh!
There was an error while loading. Please reload this page.
janniklasrose
deleted the
janniklasrose/postgres-schema-annotations-handedit
branch
August 5, 2026 13:38
eng-dev-ecosystem-bot
commented
Aug 6, 2026
Collaborator
Integration test reportCommit: e19f9b1
176 interesting tests: 131 MISS, 38 FAIL, 4 RECOVERED, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
eng-dev-ecosystem-bot
commented
Aug 6, 2026
Collaborator
Integration test reportCommit: e19f9b1
432 interesting tests: 384 MISS, 41 FAIL, 4 RECOVERED, 2 KNOWN, 1 SKIP
Top 50 slowest tests (at least 2 minutes):
|
deco-sdk-taggingBot
added a commit
that referenced
this pull request
Aug 6, 2026
## Release v1.11.0 ### CLI * Fixed `databricks repos get/update/delete` failing with `object at path "..." is not a repo` for Git-CLI-enabled folders (currently in preview), which the workspace API reports as directories rather than repos ([#6181](#6181)). * Support `dbfs:/Skills/...` paths in `databricks fs` commands, routed to the Files API. ([#6147](#6147)) ### Bundles * For jobs where `ai_runtime_task.code_source_path` is a relative path to a local directory, the directory is now packaged into a tarball (honoring `.gitignore` and `sync.include`/`sync.exclude`), uploaded during deployment, and `code_source_path` is rewritten to the uploaded workspace path. ([#6110](#6110)) * Added JSON output to `bundle init`. Running `databricks bundle init <template> -o json` now reports the files the template wrote, relative to the output directory. This lets callers that pass `--output-dir` learn where the template materialized instead of assuming the output is a single directory named after the project. The default text output is unchanged. ([#6161](#6161)) * The terraform deployment engine is deprecated and will stop working in a future version of the CLI. Setting `bundle.engine: terraform` now emits a deprecation warning. See https://docs.databricks.com/aws/en/dev-tools/bundles/direct for how to migrate to the direct deployment engine. ([#6099](#6099)) * Fixed the direct deployment engine planning a spurious `create` for an empty `grants: []` list. Terraform records no grants resource for such a list, so `bundle plan` after `bundle deployment migrate` no longer reports an action for it. Emptying a previously deployed list still revokes the grants, after which the node is dropped from the deployment state instead of being reported as unchanged forever. ([#6039](#6039)) * Fixed `bundle generate` downloading notebooks found inside a folder without their file extension. They are now exported like top-level notebooks, so a Python notebook lands as `notebook.py` instead of an extensionless file ([#6144](#6144)). * direct: `webhook_notifications.on_*` destinations on jobs, tasks, and `for_each_task` are now compared as unordered sets. Previously the Jobs API returning these lists in a different order than submitted produced a phantom diff that `bundle plan` and `bundle deploy` could never converge past, reporting `1 to change` on every run ([#6060](#6060)). * Fixed a pipeline with `allow_duplicate_names: true` never converging on the direct engine: the field is only accepted on create/update and is never returned by the pipelines GET API, so every subsequent `bundle plan` reported the pipeline as a perpetual update. ([#6076](#6076)) * direct: A local change to an input-only field (one the API accepts on write but never returns on read, e.g. pipelines' `run_as` or external locations' `skip_validation`) is no longer silently skipped when the new value coincidentally matches the field's fabricated remote value. Previously such a change could hit the `remote_already_set` shortcut and be dropped from the plan. ([#6112](#6112)) * Revert usage of RedactiveSenstiveFields (added in [#5896](#5896), released in 1.10.0) which lead to incorrect behaviour (permanent drift) for duration field in Postgres resources ([#6179](#6179)). * Document postgres resource fields in the json schema ([#6164](#6164), [#6163](#6163)). * direct: Recreating a `vector_search_indexes` resource no longer fails with "Index ... is currently pending deletion" when the backend has not yet released the index name. The create is now retried until the name becomes available. ([#6143](#6143)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.165.0 to v0.166.0. ([#6175](#6175)) * Upgrade Terraform provider to 1.124.0. ([#6174](#6174))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The postgres_* resources wrap SDK specs with fields that bundles own: the *_id components of the hierarchical resource name, parent, replace_existing, and purge_on_delete. None of these exist on the spec the annotations are inherited from, so they generated as PLACEHOLDER.
Fill them in from the upstream Create*Request docs where the semantics match, and hand-write the rest. The takeover and purge fields describe bundle behaviour rather than a single API call, and postgres_branches.parent points at source_branch instead of the upstream status.source_branch, which bundles flatten away.