Uh oh!
There was an error while loading. Please reload this page.
direct: handle remotely deleted resources - #3710
Merged
Merged
Conversation
denik
requested review from
andrewnester, anton-107, pietern and shreyas-goenka
as code ownersOctober 2, 2025 14:54
denik
enabled auto-merge
October 2, 2025 15:11
Collaborator
|
andrewnester
approved these changes
Oct 2, 2025
| // root | ||
| l := b.Config.GetLocation("resources.jobs." + name) | ||
| if l.File == "" { | ||
| // b.Config.Resources.Jobs may include a job that only exists in state but not in config |
Contributor
There was a problem hiding this comment.
Why is this possible and why only for jobs?
ContributorAuthor
There was a problem hiding this comment.
I think that's because we populate ID and URL for all resources (for summary)? I don't have a repro, this maybe triggered under specific conditions, but I've seen this particular error before.
It's only happening for jobs because this code is only run for jobs but other resources would behave the same.
Uh oh!
There was an error while loading. Please reload this page.
denik added a commit
that referenced
this pull request
May 20, 2026
## Changes
- When planning resource deletion, skip resource that were already
deleted.
- When DoDelete returns 'resource not found', consider it a success and
continue.
- Fix metadata computation not to error on resources that are deleted.
- Combine CalculatePlanFor{Deploy,Destroy} into one function.
## Why
Better plan, matches terraform.
## Tests
New acceptance test that remotely delete a job and test deploy/destroy. Merged
5 tasks
afberg pushed a commit
to afberg/cli
that referenced
this pull request
Jul 3, 2026
…emoved from config (databricks#5496) ## Why Found during a full-repo review of the CLI. With the direct engine, a resource that was deleted remotely and then removed from the bundle config could never leave the deployment state. The plan dropped its Delete entry, so the state cleanup never ran and the stale entry came back on every deploy. For pipelines and dashboards the same scenario was worse: the deploy hard-failed in metadata computation with "failed to compute relative path", and since the state never got cleaned, every following deploy failed the same way. ## Changes Before, deploying after a resource was deleted remotely and removed from config either left a stale state entry forever (jobs) or failed the deploy outright (pipelines, dashboards); now the deploy plans a delete for the stale entry, cleans it from state, and completes. - `bundle/direct/bundle_plan.go`: when planning a Delete and the remote read reports the resource is gone, keep the Delete entry instead of removing it from the plan. `Delete()` already tolerates missing resources and removes the entry from state when applied. - `bundle/deployplan/plan.go`: remove `RemoveEntry`, whose only caller was the code path above. - `bundle/deploy/metadata/compute.go`: add the missing `l.File == ""` guard to the pipelines and dashboards loops, matching the guard the jobs loop received in databricks#3710. State-only resources have no config location, and `filepath.Rel` errors on an empty path. A side effect visible in test output: `bundle destroy` with the direct engine now lists a remotely deleted resource under "The following resources will be deleted", since its state entry is now explicitly cleaned instead of silently dropped. ## Test plan - [x] New acceptance test `acceptance/bundle/resources/jobs/remote_delete/removed_from_config` covering deploy, remote delete of a job and a pipeline, config removal, deploy twice, and summary, for both engines. With the direct engine the first deploy cleans the stale entries and the second deploy and summary are clean. The terraform variant also exercises the new metadata guard, since terraform keeps the gone resources in its state. - [x] New unit test `TestComputeMetadataMutatorStateOnlyResources` verifying state-only jobs, pipelines, and dashboards are skipped without error. - [x] Unit tests for `bundle/direct`, `bundle/deployplan`, and `bundle/deploy/metadata`. - [x] Existing acceptance tests around the changed paths (`jobs/remote_delete`, `volumes/remote-delete`, `volumes/remote-change-name`, `vector_search_indexes/drift`, `permissions`, `deployment/bind`, `deployment/unbind`, `bundle/destroy`, `bundle/generate`, `bundle/invariant`, `clusters`, `quality_monitors`, `registered_models`, synced tables). Only `jobs/remote_delete/destroy` changed output (direct destroy now lists the gone job) and was regenerated with `-update`. - [x] `./task fmt-q`, `./task lint-q`, `./task checks`. This pull request and its description were written by Isaac.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Why
Better plan, matches terraform.
Tests
New acceptance test that remotely delete a job and test deploy/destroy.