Skip to content

[direct] Implement bundle summary/run/open and resource deletion - #3162

Merged
denik merged 16 commits into
mainfrom
denik/terranova-load
Jul 3, 2025
Merged

[direct] Implement bundle summary/run/open and resource deletion#3162
denik merged 16 commits into
mainfrom
denik/terranova-load

Conversation

@denik

@denikdenik commented Jul 1, 2025

Copy link
Copy Markdown
Contributor

Changes

  • Fix resource deletion in direct backend: resource removed from the config are now deleted.
  • Implement bundle summary/run/open in direct backend.

Depends on

Tests

Enabled a bunch of tests & added more precise comments on remaining disabled.

@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 16:30 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/terranova-load branch from b57fb5e to 570e05dCompareJuly 1, 2025 16:30
@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 16:30 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

Run: 16045388215

Env✅‌pass❌‌FAIL💥‌PANIC🔄‌flaky🙈‌skip🤯‌MISS
❌‌aws linux293813732
❌‌aws windows29771372
✅‌aws-ucws linux408273
✅‌aws-ucws windows409272
✅‌azure linux306372
✅‌azure windows307371
✅‌azure-ucws linux410270
✅‌azure-ucws windows411269
✅‌gcp linux305374
✅‌gcp windows306373
16 failing tests:
Test Nameaws linuxaws windows
TestAccept✅‌pass❌‌FAIL
TestAccept/bundle/deployment/bind/dashboard✅‌pass🔄‌flaky
TestAccept/bundle/resources/pipelines✅‌pass❌‌FAIL
TestAccept/bundle/resources/pipelines/DATABRICKS_CLI_DEPLOYMENT=terraform✅‌pass❌‌FAIL
TestFetchRepositoryInfoAPI_FromRepo❌‌FAIL❌‌FAIL
TestLock✅‌pass❌‌FAIL
TestReposCreateWithProvider❌‌FAIL✅‌pass
TestReposCreateWithoutProvider❌‌FAIL✅‌pass
TestReposDeleteByID❌‌FAIL✅‌pass
TestReposGet❌‌FAIL✅‌pass
TestReposUpdate❌‌FAIL✅‌pass
TestSyncEnsureRemotePathIsUsableIfRepoDoesntExist💥‌PANIC✅‌pass
TestSyncEnsureRemotePathIsUsableIfRepoExists🤯‌MISS❌‌FAIL
TestSyncEnsureRemotePathIsUsableInWorkspace🤯‌MISS✅‌pass
TestSyncIncrementalFileOverwritesFolder❌‌FAIL✅‌pass
TestSyncNestedFolderDoesntFailOnNonEmptyDirectory❌‌FAIL❌‌FAIL

@denik
denikforce-pushed the denik/move-terraform-load branch 6 times, most recently from a06ba9f to 97f28f0CompareJuly 2, 2025 11:34
@denik
denikforce-pushed the denik/terranova-load branch from 570e05d to 4fa1402CompareJuly 2, 2025 14:02
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 14:03 — with GitHub Actions Inactive
@denikdenik changed the title WIP - loading resources id into bundle in direct deployment[direct] Implement bundle summary/run/open and resource deletionJul 2, 2025
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 14:06 — with GitHub Actions Inactive
@denik
denik marked this pull request as ready for review July 2, 2025 14:10
Base automatically changed from denik/move-terraform-load to mainJuly 3, 2025 07:46
denik added 10 commits July 3, 2025 09:48
clean up
clean up updated
wip
break dependency of statemgmt on deploy/terraform
fixes
add state_export.go
update secret scopes test
move ResourceState to its own package
create resources.X if it does not exist
expand modified_status to test absence of resources: as well
create resources & resources.section if missing
restore empty resources{}
Move terraform.Load() to statemgmt.Load()
Pure move, no functional changes.
load state into bundle - run/basic passes
enable run/no-state
Enable integration_whl on direct-exp
WIP
@denik
denikforce-pushed the denik/terranova-load branch from 3161e1d to 95bc4c1CompareJuly 3, 2025 07:48
@denik
deniktemporarily deployed to test-trigger-is July 3, 2025 07:48 — with GitHub Actions Inactive
diags.AppendErrorf("internal error: cannot get config for group=%v name=%v", node.Group, node.Name)
return
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary now and not before? (useful to capture in a comment here as well)

I suspect it's because the state loader now populates configs (and IDs) regardless of presence in the config.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deletes were not working before in all scenarios but we did not have enough tests enabled to catch that.

This part assumes that config is always present which is of course not the case when config was deleted:

 config, ok := b.GetResourceConfig(node.Group, node.Name)
if !ok {
diags.AppendErrorf("internal error: cannot get config for %s", node)

panic("direct deployment required")
}

err := b.OpenResourceDatabase(ctx)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this call idempotent?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes (in this PR).

Comment threadcmd/bundle/open.go
@denik
deniktemporarily deployed to test-trigger-is July 3, 2025 07:54 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is July 3, 2025 08:01 — with GitHub Actions Inactive
@denik
denik requested a review from pieternJuly 3, 2025 08:11
Comment threadbundle/terranova/apply.go Outdated
var config any

if node.ActionType == deployplan.ActionTypeDelete {
config = nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With config already beingnil (zero-initialized), a switch/case could improve readability here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, not sure about switch but cleaned up a bit 3460742

Comment threadbundle/terranova/tnstate/state.go Outdated
return fmt.Errorf("Already read state %v, cannot open %v", db.Path, path)
}

db.Path = path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: to make this more robust, set path only after successfully reading and deserializing the state.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense 1fd9d32

@denik
deniktemporarily deployed to test-trigger-is July 3, 2025 08:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is July 3, 2025 08:30 — with GitHub Actions Inactive
@denik
denik merged commit d1c34da into mainJul 3, 2025
@denik
denik deleted the denik/terranova-load branch July 3, 2025 09:27
denik added a commit that referenced this pull request Jul 7, 2025
denik added a commit that referenced this pull request Jul 9, 2025
It requires "bundle summary" which we have since #3162
github-merge-queueBot pushed a commit that referenced this pull request Jul 9, 2025
It requires "bundle summary" which we have since
#3162
denik added a commit that referenced this pull request May 20, 2026
## Changes
- Fix resource deletion in direct backend: resource removed from the
config are now deleted.
- Implement bundle summary/run/open in direct backend.
## Depends on
- #3134
- #3161
## Tests
Enabled a bunch of tests & added more precise comments on remaining
disabled.
denik added a commit that referenced this pull request May 20, 2026
denik added a commit that referenced this pull request May 20, 2026
It requires "bundle summary" which we have since
#3162
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@eng-dev-ecosystem-bot@pietern