Skip to content

postgres: Support replace_existing on databases and roles - #5803

Merged
pietern merged 19 commits into
mainfrom
postgres-replace-existing-database-role
Jul 9, 2026
Merged

postgres: Support replace_existing on databases and roles#5803
pietern merged 19 commits into
mainfrom
postgres-replace-existing-database-role

Conversation

@pietern

@pieternpietern commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Extends the replace_existing pattern from postgres_branches / postgres_endpoints (#5264) to postgres_databases and postgres_roles. When true, the create call takes over a database or role that already exists on a Lakebase branch — for example a role inherited from the parent branch — instead of failing with ALREADY_EXISTS.

The flag is input-only; the GET API never echoes it, so RemapState pins it to false and an ignore_local_changes rule makes toggling it after create a no-op.

Acceptance tests cover the deploy → unbind → re-deploy adoption flow on both engines and assert replace_existing=true is sent.

This pull request and its description were written by Isaac.

pietern added 14 commits June 30, 2026 12:50
# Conflicts:
#	libs/structs/structwalk/walktype_test.go
Bump the pinned Databricks Terraform provider from v1.118.0 to v1.120.0,
regenerate bundle/internal/tf/schema (and its ProviderVersion/checksums),
and refresh the affected acceptance goldens.
Co-authored-by: Isaac
Co-authored-by: Isaac
Extend the replace_existing pattern already used by postgres_branches and
postgres_endpoints to databases and roles: a bundle can take over a database
or role that already exists on a Lakebase branch instead of failing with
ALREADY_EXISTS.
The Terraform provider does not support replace_existing for databases/roles
yet, so the new acceptance tests are locked to the direct engine for now.
Co-authored-by: Isaac
The Terraform provider 1.120.0 bump adds replace_existing to the
postgres_database and postgres_role schemas, so the new acceptance tests can
run on terraform in addition to direct. The recorded request set diverges
between engines (terraform omits the extra GET polls and includes parent in the
body), so the deploy requests are split into per-engine out.requests.deploy.<engine>.json.
Co-authored-by: Isaac
…ting-database-role
# Conflicts:
#	NEXT_CHANGELOG.md
#	bundle/terraform_dabs_map/generated.go
@pietern
pieterntemporarily deployed to test-trigger-is July 2, 2026 12:52 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 2, 2026 12:52 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: b405b1d

Run: 28932252488

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
🔄​aws linux24422810667:42
💚​aws windows4423210647:09
💚​aws-ucws linux443149845:16
💚​aws-ucws windows443169827:08
💚​azure linux4423010655:02
💚​azure windows4423210637:09
💚​azure-ucws linux443169816:06
🔄​azure-ucws windows31431897911:18
💚​gcp linux4422910674:47
💚​gcp windows4423110657:25
10 interesting tests: 5 flaky, 4 SKIP, 1 RECOVERED
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestSecretsPutSecretBytesValue🔄​f✅​p🙈​s🙈​s✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestSecretsPutSecretStringValue🔄​f✅​p🙈​s🙈​s✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R
🔄​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R
🔄​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R💚​R💚​R💚​R🔄​f💚​R💚​R
Top 11 slowest tests (at least 2 minutes):
durationenvtestname
6:17gcp windowsTestAccept
6:15azure windowsTestAccept
6:13aws windowsTestAccept
6:12azure-ucws windowsTestAccept
6:09aws-ucws windowsTestAccept
2:55gcp linuxTestAccept
2:53aws linuxTestAccept
2:53azure linuxTestAccept
2:47azure-ucws linuxTestAccept
2:46aws linuxTestSecretsPutSecretStringValue
2:44aws-ucws linuxTestAccept

…ting-database-role
# Conflicts:
#	NEXT_CHANGELOG.md
@pietern
pieterntemporarily deployed to test-trigger-is July 7, 2026 12:18 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 7, 2026 12:18 — with GitHub Actions Inactive
…hanges
Revert branch/endpoint replace_existing annotations to PLACEHOLDER so this PR
only carries new descriptions for postgres_databases and postgres_roles.
Co-authored-by: Isaac
@pietern
pieterntemporarily deployed to test-trigger-is July 7, 2026 14:24 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 7, 2026 14:24 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 8, 2026 09:27 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 8, 2026 09:27 — with GitHub Actions Inactive
@pieternpietern changed the title postgres: Support replace_existing on databases and rolespostgres: Support replace_existing on databases and rolesJul 8, 2026
@pietern
pietern marked this pull request as ready for review July 8, 2026 09:40
Updating deployment state...
Deployment complete!

=== Confirm the role is under management again

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to see get-role after unbind to observe the difference

The unbind-staging flow works identically on cloud (unbind only edits local
bundle state; the resource stays on the branch), so these tests can run against
the real Lakebase API. Drop the Cloud = false override (empty test.toml removed;
inherits Cloud = true from the parent) and refresh out.test.toml.
Verified on aws-prod-ucws: both engines adopt the existing database/role with
replace_existing, no golden drift.
Co-authored-by: Isaac
@pietern
pieterntemporarily deployed to test-trigger-is July 9, 2026 09:14 — with GitHub Actions Inactive
@pietern
pieterntemporarily deployed to test-trigger-is July 9, 2026 09:14 — with GitHub Actions Inactive
@pietern
pietern merged commit 681f8b0 into mainJul 9, 2026
22 checks passed
@pietern
pietern deleted the postgres-replace-existing-database-role branch July 9, 2026 09:38
deco-sdk-taggingBot added a commit that referenced this pull request Jul 9, 2026
## Release v1.7.0
### CLI
* An explicitly selected profile (`--profile` or a bundle's `workspace.profile`) now takes precedence over auth environment variables (`DATABRICKS_HOST`, `DATABRICKS_TOKEN`, etc.) instead of being silently shadowed by them; env vars still fill auth fields the profile leaves empty ([#5096](#5096)).
* Fix intermittent crashes when processing pages from API calls ([#5815](#5815)).
### Bundles
* direct: add basic version of job_runs resource (experimental) ([#5603](#5603)).
* Fix permissions added to a job or pipeline by a Python (PyDABs) mutator failing to deploy with "must have exactly one owner"; the deploying identity is now set as owner, matching resources whose permissions are declared in YAML ([#5821](#5821)).
* Remove duplicate enum values for jsonschema.json ([#5839](#5839)).
* direct: volumes: support `volume_path` property ([#5550](#5550)).
* direct: Fix deploy bug when a `postgres_projects`, `postgres_branches`, or `postgres_endpoints` field is set to its zero value (e.g. `enable_pg_native_login: false`, `replace_existing: false`) ([#5782](#5782)).
* `bundle run --only` help now documents the `+` modifier syntax: prefix a task key with `+` to also run its upstream tasks, or suffix it with `+` for downstream tasks ([#5760](#5760)).
* direct: Recognize UC-managed catalog and schema property defaults to avoid unnecessary drift ([#5865](#5865) & [#5870](#5870)).
* Fix `bundle deploy --select <resource>` skipping the resource's grants and permissions; they are now applied as part of the selected resource ([#5852](#5852)).
* Support `purge_on_delete: true` on `postgres_branches` so bundles can hard-delete a Lakebase branch on destroy (skipping the soft-delete retention window) ([#5801](#5801)).
* Support `replace_existing: true` on `postgres_databases` and `postgres_roles` so bundles can take over a database or role that already exists on a Lakebase branch instead of failing with `ALREADY_EXISTS` ([#5803](#5803)).
### Dependency updates
* Bump databricks-sdk-go to v0.154.0 ([#5855](#5855)).
* Bump terraform-provider to 1.121.0 ([#5857](#5857)).
* Bump OpenTelemetry dependencies to v1.44.0 to address [CVE-2026-41178](GHSA-5wrp-cwcj-q835) ([#5873](#5873)).
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 681f8b0

Run: 29009027270

Env❌​FAIL🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
🔄​aws linux334527100641:13
🔄​aws windows444482101952:03
❌​aws-ucws linux91452988817184:15
❌​aws-ucws windows91752927835193:17
🔄​azure linux444519100743:32
🔄​azure windows644474102056:12
🔄​azure-ucws linux352894852141:52
🔄​azure-ucws windows462834870144:55
💚​gcp linux44514101248:52
🔄​gcp windows444467102561:25
29 interesting tests: 12 flaky, 9 FAIL, 5 RECOVERED, 2 SKIP, 1 KNOWN
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
🟨​TestAccept🔄​f💚​R🟨​K🟨​K💚​R💚​R🔄​f💚​R💚​R💚​R
🔄​TestAccept/bundle/generate/pipeline_and_deploy✅​p🔄​f🔄​f🔄​f✅​p🔄​f🔄​f🔄​f✅​p🔄​f
🔄​TestAccept/bundle/generate/pipeline_and_deploy/DATABRICKS_BUNDLE_ENGINE=direct✅​p🔄​f🔄​f🔄​f✅​p🔄​f🔄​f🔄​f✅​p🔄​f
🔄​TestAccept/bundle/generate/python_job_and_deploy🔄​f🔄​f🔄​f🔄​f🔄​f🔄​f✅​p🔄​f✅​p🔄​f
🔄​TestAccept/bundle/generate/python_job_and_deploy/DATABRICKS_BUNDLE_ENGINE=direct🔄​f🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/generate/python_job_and_deploy/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p🔄​f🔄​f🔄​f🔄​f✅​p🔄​f✅​p✅​p
💚​TestAccept/bundle/invariant/no_drift🙈​S🙈​S💚​R💚​R🙈​S🙈​S💚​R💚​R🙈​S🙈​S
❌​TestAccept/bundle/resources/postgres_endpoints/basic🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling🙈​s🙈​s❌​F❌​F🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=direct❌​F❌​F
❌​TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=terraform❌​F❌​F
🔄​TestAccept/bundle/resources/postgres_synced_tables/recreate🙈​s🙈​s✅​p🔄​f🙈​s🙈​s🙈​s🙈​s🙈​s🙈​s
🔄​TestAccept/bundle/resources/postgres_synced_tables/recreate/DATABRICKS_BUNDLE_ENGINE=direct✅​p🔄​f
💚​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S💚​R💚​R🙈​S🙈​S💚​R💚​R🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestAccept/bundle/templates/default-python/combinations/classic✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=no/READPLAN=✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.10✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestSyncIncrementalSyncFileToPythonNotebook✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p
💚​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
💚​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
Top 50 slowest tests (at least 2 minutes):
durationenvtestname
11:57gcp windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:29gcp linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:19gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:52aws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:36aws-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:21azure-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:20gcp windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
9:19azure linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
9:05aws-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:58aws-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:56aws windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
8:46azure windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:45aws-ucws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
8:26aws-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
8:21azure-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:12aws windowsTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:44aws-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:40aws linuxTestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:18azure-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:54gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:49gcp windowsTestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:17aws-ucws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
5:52azure-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:49azure-ucws windowsTestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:38aws windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:29aws-ucws linuxTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=no/PY=no/READPLAN=1
5:24azure-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:16aws-ucws windowsTestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=terraform
5:14azure-ucws windowsTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:14aws-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:12gcp linuxTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:12azure-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:12azure windowsTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:11azure linuxTestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:08aws-ucws windowsTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
5:07aws windowsTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:06gcp linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:01azure-ucws linuxTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
4:58azure-ucws linuxTestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
4:57azure-ucws windowsTestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=yes/READPLAN=
4:51aws-ucws windowsTestAccept/bundle/resources/postgres_databases/update/DATABRICKS_BUNDLE_ENGINE=terraform
4:50aws-ucws linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:46aws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:43gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:41aws-ucws linuxTestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:38aws linuxTestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:35gcp windowsTestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:35aws-ucws windowsTestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
4:32gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:31gcp linuxTestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pietern@eng-dev-ecosystem-bot@janniklasrose