Skip to content

Move terraform.Load() to statemgmt.Load() - #3161

Merged
denik merged 1 commit into
mainfrom
denik/move-terraform-load
Jul 3, 2025
Merged

Move terraform.Load() to statemgmt.Load()#3161
denik merged 1 commit into
mainfrom
denik/move-terraform-load

Conversation

@denik

@denikdenik commented Jul 1, 2025

Copy link
Copy Markdown
Contributor

Changes

Move terraform.Load to statemgmt.Load
Move terraform.TerraformToBundle to statemgmt.StateToBundle.
Move related unit tests.

Pure move, no functional changes.

Stacked on top of #3134

Why

Preparation for making it work with direct deployment.

@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 15:41 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/improve-terraform-load branch from 878520f to fd3c594CompareJuly 1, 2025 15:43
@denik
denikforce-pushed the denik/move-terraform-load branch from 68cd7d2 to 99d2e3fCompareJuly 1, 2025 15:44
@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 15:45 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/move-terraform-load branch from 99d2e3f to 4eefba3CompareJuly 1, 2025 15:58
@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 15:58 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

Run: 16024008342

Env✅‌pass❌‌FAIL💥‌PANIC🔄‌flaky🙈‌skip🤯‌MISS
❌‌aws linux23828135744
❌‌aws windows23215235564
✅‌aws-ucws linux399273
❌‌aws-ucws windows388312728
✅‌azure linux297372
✅‌azure windows298371
💥‌azure-ucws linux390127010
🔄‌azure-ucws windows3984269
✅‌gcp linux296374
✅‌gcp windows297373
93 failing tests:
Test Nameaws linuxaws windowsaws-ucws windowsazure-ucws linuxazure-ucws windows
TestAccept🤯‌MISS🤯‌MISS✅‌pass✅‌pass🔄‌flaky
TestAccept/bundle/deploy/dashboard/detect-change❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/dashboard/simple✅‌pass❌‌FAIL✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/dashboard/simple_syncroot✅‌pass❌‌FAIL✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/files/no-snapshot-sync🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_CLI_DEPLOYMENT=direct-exp🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_CLI_DEPLOYMENT=terraform🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/jobs/fail-on-active-runs🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/jobs/fail-on-active-runs/DATABRICKS_CLI_DEPLOYMENT=direct-exp🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/jobs/fail-on-active-runs/DATABRICKS_CLI_DEPLOYMENT=terraform🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/jobs/shared-root-path🤯‌MISS🤯‌MISS✅‌pass✅‌pass🔄‌flaky
TestAccept/bundle/deploy/jobs/shared-root-path/DATABRICKS_CLI_DEPLOYMENT=direct-exp🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deploy/jobs/shared-root-path/DATABRICKS_CLI_DEPLOYMENT=terraform🤯‌MISS🤯‌MISS✅‌pass✅‌pass🔄‌flaky
TestAccept/bundle/deploy/mlops-stacks❌‌FAIL✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deployment/bind/dashboard❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestAccept/bundle/deployment/bind/dashboard/recreation❌‌FAIL✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/local_state_staleness🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/local_state_staleness/DATABRICKS_CLI_DEPLOYMENT=direct-exp🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/local_state_staleness/DATABRICKS_CLI_DEPLOYMENT=terraform🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/resources/pipelines🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/resources/pipelines/DATABRICKS_CLI_DEPLOYMENT=direct-exp🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/resources/pipelines/DATABRICKS_CLI_DEPLOYMENT=terraform🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=no/NBOOK=yes/PY=yes🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=no/PY=yes🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=yes/PY=no🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=yes/PY=yes💥‌PANIC✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=no/PY=no🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=no/PY=yes🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=yes🤯‌MISS✅‌pass✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=no/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=no/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=no🤯‌MISS💥‌PANIC✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=no/NBOOK=no/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=no/NBOOK=no/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=no/NBOOK=yes/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=no/NBOOK=yes/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=no/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=no/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=yes/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=yes/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=no/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=no/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=no/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=no/PY=yes🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=no🤯‌MISS🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=yes🙈‌skip🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic❌‌FAIL🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.10✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.11❌‌FAIL🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.12✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.13✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.9✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/basic✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/basic/DATABRICKS_CLI_DEPLOYMENT=direct-exp✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/basic/DATABRICKS_CLI_DEPLOYMENT=terraform✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/error✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/error/DATABRICKS_CLI_DEPLOYMENT=direct-exp✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/error/DATABRICKS_CLI_DEPLOYMENT=terraform✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/pipeline-crud✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_CLI_DEPLOYMENT=direct-exp✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_CLI_DEPLOYMENT=terraform✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/workspace-file-io❌‌FAIL🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=direct-exp❌‌FAIL🤯‌MISS✅‌pass✅‌pass✅‌pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=terraform✅‌pass🤯‌MISS✅‌pass✅‌pass✅‌pass
TestFetchRepositoryInfoAPI_FromRepo❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestGenerateFromExistingJobAndDeploy❌‌FAIL✅‌pass✅‌pass✅‌pass✅‌pass
TestInprocessMode🤯‌MISS🤯‌MISS🙈‌skip🙈‌skip🙈‌skip
TestLock❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestReposCreateWithProvider❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestReposCreateWithoutProvider❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestReposDeleteByID❌‌FAIL❌‌FAIL❌‌FAIL✅‌pass✅‌pass
TestReposDeleteByPath❌‌FAIL❌‌FAIL❌‌FAIL✅‌pass✅‌pass
TestReposGet❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestReposUpdate❌‌FAIL❌‌FAIL❌‌FAIL✅‌pass✅‌pass
TestSparkJarTaskDeployAndRunOnWorkspace❌‌FAIL✅‌pass✅‌pass✅‌pass✅‌pass
TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS❌‌FAIL✅‌pass✅‌pass✅‌pass✅‌pass
TestSyncEnsureRemotePathIsUsableIfRepoDoesntExist✅‌pass🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncEnsureRemotePathIsUsableIfRepoExists❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncEnsureRemotePathIsUsableInWorkspace✅‌pass🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncFullFileSync❌‌FAIL❌‌FAIL✅‌pass✅‌pass✅‌pass
TestSyncIncrementalFileOverwritesFolder❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncIncrementalFileSync✅‌pass❌‌FAIL✅‌pass💥‌PANIC✅‌pass
TestSyncIncrementalSyncFileToPythonNotebook❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncIncrementalSyncPythonNotebookDelete❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncIncrementalSyncPythonNotebookToFile❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass
TestSyncNestedFolderDoesntFailOnNonEmptyDirectory❌‌FAIL💥‌PANIC💥‌PANIC🤯‌MISS🔄‌flaky
TestSyncNestedFolderSync❌‌FAIL❌‌FAIL✅‌pass🤯‌MISS✅‌pass
TestSyncNestedSpacePlusAndHashAreEscapedSync❌‌FAIL🤯‌MISS🤯‌MISS🤯‌MISS✅‌pass

return "statemgmt.Load"
}

func (l *load) Apply(ctx context.Context, b *bundle.Bundle) diag.Diagnostics {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still TF-specific. I expected the StateToBundle function to move here, but I still keep separate mutators for the different deployment modes. Doesn't that make more sense?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is purely a move. The mutator will be refactored to handle both terraform and terranova case.

It's easier to have single mutator that handles both, less changes are needed in cmd/bundle/*.go

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still want to keep the different implementations in different packages?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actual logic that converts tfstate or resources.json to neutral representation will leave in corresponding package. The mutator acts as aggregator, so it can take dependency on both terraform and terranova to call the right functions.

Base automatically changed from denik/improve-terraform-load to mainJuly 2, 2025 09:21
@denik
denik requested a review from pieternJuly 2, 2025 09:22
@denik
denikforce-pushed the denik/move-terraform-load branch from ba86b34 to f3183b4CompareJuly 2, 2025 09:24
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 09:24 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/move-terraform-load branch from f3183b4 to 0f4fae1CompareJuly 2, 2025 09:26
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 09:26 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/move-terraform-load branch from 0f4fae1 to 75d651fCompareJuly 2, 2025 09:31
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 09:31 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/move-terraform-load branch from 75d651f to a06ba9fCompareJuly 2, 2025 11:00
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 11:00 — with GitHub Actions Inactive
Pure move, no functional changes.
@denik
denikforce-pushed the denik/move-terraform-load branch from a06ba9f to 97f28f0CompareJuly 2, 2025 11:34
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 11:34 — with GitHub Actions Inactive
@denik
denik merged commit 3356a97 into mainJul 3, 2025
@denik
denik deleted the denik/move-terraform-load branch July 3, 2025 07:46
denik added a commit that referenced this pull request Jul 3, 2025
## Changes
- Fix resource deletion in direct backend: resource removed from the
config are now deleted.
- Implement bundle summary/run/open in direct backend.
## Depends on
- #3134
- #3161
## Tests
Enabled a bunch of tests & added more precise comments on remaining
disabled.
denik added a commit that referenced this pull request May 20, 2026
## Changes
Move terraform.Load to statemgmt.Load
Move terraform.TerraformToBundle to statemgmt.StateToBundle.
Move related unit tests.
Pure move, no functional changes.
Stacked on top of #3134
## Why
Preparation for making it work with direct deployment.
denik added a commit that referenced this pull request May 20, 2026
## Changes
- Fix resource deletion in direct backend: resource removed from the
config are now deleted.
- Implement bundle summary/run/open in direct backend.
## Depends on
- #3134
- #3161
## Tests
Enabled a bunch of tests & added more precise comments on remaining
disabled.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@eng-dev-ecosystem-bot@pietern