Uh oh!
There was an error while loading. Please reload this page.
postgres: Support purge_on_delete on postgres_projects - #5414
Merged
Conversation
Collaborator
Commit: 6d1b897 |
Adds support for `purge_on_delete` on Lakebase `postgres_projects` so bundles can hard-delete a project on destroy. The flag is passed to the DeleteProject API call as `?purge=true`; when unset, the backend performs a soft delete that can be undone via `databricks postgres undelete-project` within the project's retention window. The field is input-only — it is not modeled by the backend resource for projects, and the GET API never returns it. We store it in state purely so DoDelete can apply it on destroy: by that point the configuration for the resource is gone, so state is the only place it can live. PrepareState preserves `input.ForceSendFields` so the structdiff comparison correctly distinguishes "explicit false" from the fictional remote zero — otherwise toggling `true -> false` would be classified as no change, state would stay `true`, and the next destroy would still emit `?purge=true`. DoUpdate strips `purge_on_delete` from the API field mask so a state-only flip doesn't fire an unnecessary remote write. Acceptance tests under `acceptance/bundle/resources/postgres_projects/`: - `purge_on_delete/`: deploys a `hard_delete` and a `soft_delete` project side by side and asserts the destroy emits `?purge=true` and a plain DELETE respectively, on both engines. - `purge_on_delete_transitions/`: direct-engine only. Walks `purge_on_delete` through unset -> true -> false -> unset and records the persisted value at each step; final destroy is a plain DELETE. Regression coverage for the FSF-preservation fix. Manually verified against dogfood with ephemeral projects on both engines (deploy -> flip -> destroy; GET on project/branches/endpoints in soft vs hard cases; native `postgres undelete-project` restoration semantics — captured in DECO-27233). Co-authored-by: Isaac
denik
reviewed
Jun 2, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| @@ -0,0 +1,4 @@ | |||
| # Direct engine only: this test exercises the FSF preservation in | |||
| # PrepareState and direct's plan/diff classification when the user flips | |||
| # purge_on_delete. Terraform has its own provider-managed state lifecycle. | |||
Contributor
There was a problem hiding this comment.
but destroy requests should match between TF and direct, right? so could still make sense to run part of of these tests on TF to record destroy requests and compare.
ContributorAuthor
There was a problem hiding this comment.
Cross-engine destroy parity is already covered by the sibling purge_on_delete/ test, which records the destroy on both engines side by side. This test is about the direct-engine state lifecycle specifically (FSF + plan classification), which TF doesn't share — the TF provider manages its own state shape. Keeping it direct-only avoids divergent state captures.
denik
reviewed
Jun 2, 2026
Uh oh!
There was an error while loading. Please reload this page.
- script: drop --keep flags on print_requests.py (was double-printing deploy requests in destroy.json) and redirect cleanup-noise to LOG.delete-project so it's visible under `go test -v`. - purge_on_delete_transitions/script: replace inline jq with the `gron.py | grep` idiom used elsewhere; unset surfaces as "(unset)". - DoUpdate: keep the change-list approach. Tried a static spec mask but the API rejects "*" (expands to nested attrs the body must populate) and rejects fields in the mask that aren't also in the body, so the mask has to mirror what the user actually set. Confirmed on dogfood. Co-authored-by: Isaac
denik
approved these changes
Jun 3, 2026
pietern
enabled auto-merge
June 3, 2026 11:17
Uh oh!
There was an error while loading. Please reload this page.
eng-dev-ecosystem-bot
commented
Jun 3, 2026
Collaborator
Commit: a5f9263 |
deco-sdk-taggingBot
added a commit
that referenced
this pull request
Jun 4, 2026
## Release v1.2.0 ### CLI * `experimental open` now opens every DABs resource type that has a workspace URL, picking up `catalogs`, `schemas`, `volumes`, `database_instances`, `database_catalogs`, `synced_database_tables`, `postgres_catalogs`, `postgres_synced_tables`, `quality_monitors`, `vector_search_endpoints`, and `vector_search_indexes` ([#5346](#5346)). ### Bundles * Retry transient HTTP 5xx and 408 errors in direct deployment engine ([#5349](#5349), [#5364](#5364)). * Preserve `.designer.ipynb` suffix when translating notebook task paths so Lakeflow Designer files referenced from a `notebook_task` resolve correctly in the workspace ([#5370](#5370)). * Fix script output dropping last line without trailing newline ([#4995](#4995)). * engine/direct: Add `--select` flag to `bundle plan` and `bundle deploy` to plan/deploy a subset of resources (e.g. `--select my_job` or `--select jobs.my_job`); resources referenced by the selection are included transitively ([#5413](#5413)). * Support `purge_on_delete: true` on `postgres_projects` so bundles can hard-delete a Lakebase project on destroy (skipping the soft-delete retention window) ([#5414](#5414)). * Support terraform references in direct engine ([#5392](#5392)) * Support lifecycle.started for SQL warehouses ([#5348](#5348)) ### Dependency updates * Bump Go toolchain to 1.26.4 ([#5420](#5420)). * Bump `github.com/databricks/databricks-sdk-go` from v0.136.0 to v0.141.0 ([#5361](#5361)) * Bump Terraform provider from v1.115.0 to v1.117.0 ([#5421](#5421))
github-merge-queueBot
pushed a commit
that referenced
this pull request
Jun 4, 2026
A cloud run of the `postgres_projects/purge_on_delete` acceptance test (added in #5414) showed that the backend materializes `default_endpoint_settings` (`1`/`1`/`86400s`) and `history_retention_duration` (`604800s`) in the project status even when the spec omits them. The local testserver left these empty, diverging from cloud. `PostgresProjectCreate` now fills in these backend defaults when the spec doesn't provide them, with any spec-provided values overriding field-by-field. This brings the local test in line with the cloud golden output. This pull request and its description were written by Isaac.
Chambras pushed a commit
to Chambras/newDatabrickscli
that referenced
this pull request
Jul 6, 2026
…ope to test files (databricks#5422) Three acceptance-test idioms surfaced in databricks#5414 review that weren't captured in `.agent/rules/testing.md`: - Prefer `gron.py | grep` over `jq` for single-value lookups. - Don't pass `--keep` to `print_requests.py` if a later call follows. - Route noisy command output to `LOG.<name>` — `&> LOG.deploy` + `contains.py '!panic'`, or `2>>LOG.<name>` for cleanup stderr. Added `paths:` frontmatter so the rules auto-load when Claude reads test files. This pull request and its description were written by Isaac.
pietern added a commit
that referenced
this pull request
Jul 9, 2026
Adds `purge_on_delete` to Lakebase `postgres_branches`, mirroring `postgres_projects` (#5414). When `true`, destroy calls `DeleteBranch` with `?purge=true` (hard delete); otherwise the branch is soft-deleted and recoverable within its retention window. The flag is input-only, kept in state so `DoDelete` can apply it on destroy. `DoUpdate` strips it from the field mask so toggling it is a state-only refresh. Acceptance tests assert the `?purge=true` vs plain `DELETE` contrast on both engines. This pull request and its description were written by Isaac.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for
purge_on_deleteon Lakebasepostgres_projectsso bundles can hard-delete a project on destroy. The flag is passed to the DeleteProject API call as?purge=true; when unset, the backend performs a soft delete that can be undone viadatabricks postgres undelete-projectwithin the project's retention window.The field is input-only — it is not modeled by the backend resource for projects, and the GET API never returns it. We store it in state purely so
DoDeletecan apply it on destroy: by that point the configuration for the resource is gone, so state is the only place it can live.PrepareStatepreservesinput.ForceSendFieldsso the structdiff comparison correctly distinguishes "explicit false" from the fictional remote zero — otherwise togglingtrue → falsewould be classified as no change, state would staytrue, and the next destroy would still emit?purge=true.DoUpdatestripspurge_on_deletefrom the API field mask so a state-only flip doesn't fire an unnecessary remote write.Acceptance tests under
acceptance/bundle/resources/postgres_projects/:purge_on_delete/: deploys ahard_deleteand asoft_deleteproject side by side and asserts the destroy emits?purge=trueand a plainDELETErespectively, on both engines.purge_on_delete_transitions/: direct-engine only. Walkspurge_on_deletethrough unset → true → false → unset and records the persisted value at each step; final destroy is a plainDELETE. Regression coverage for the FSF-preservation fix.Test plan
databricks postgres undelete-projectrestoration semantics confirmed end-to-end (project + implicit production branch + primary endpoint return; user-created sub-resources are cascade-deleted and not restored — captured in DECO-27233).This pull request and its description were written by Isaac.