Skip to content

Breakout variable lookup into separate files and tests - #1921

Merged
pietern merged 7 commits into
mainfrom
variable-lookup-denormalize
Nov 21, 2024
Merged

Breakout variable lookup into separate files and tests#1921
pietern merged 7 commits into
mainfrom
variable-lookup-denormalize

Conversation

@pietern

Copy link
Copy Markdown
Contributor

Changes

While looking into adding variable lookups for notification destinations (API), I found the codegen approach for different classes of variable lookups a bit complex. The template had a custom field override (for service principals), the package had an override for the cluster lookup, and it didn't produce tests.

The notification destinations API uses a default page size of 20 for listing. I want to use a larger page size to limit the number of API calls, so that would imply another customization on the template or a manual override.

This code being rather mechanical, I used copilot to produce all instances of the resolvers and their tests (after writing one of them manually).

Tests

  • Unit tests pass
  • Manual confirmation that lookups of warehouses still work

@pieternpietern changed the title Breakout variable lookup into dedicated files and testsBreakout variable lookup into separate files and testsNov 20, 2024
@github-actions

Copy link
Copy Markdown
Contributor

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 1921
  • Commit SHA: e8e89b0cbf7d1adb0802ba481b9dbc801a23c452

Checks will be approved automatically on success.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Test Details: go/deco-tests/11942955751

@pietern

Copy link
Copy Markdown
ContributorAuthor

Integration test failures because of #1924. No need to re-run.

@pietern
pietern merged commit 14fe03d into mainNov 21, 2024
@pietern
pietern deleted the variable-lookup-denormalize branch November 21, 2024 10:28
pietern added a commit that referenced this pull request Nov 21, 2024
## Changes
Add support for notification destinations in variable lookups.
More information:
https://docs.databricks.com/en/admin/workspace-settings/notification-destinations.html
Depends on #1921.
## Tests
* New unit test
* Manually confirmed that the lookup works
pietern added a commit that referenced this pull request Dec 5, 2024
**New features for Databricks Asset Bundles:**
This release adds support for managing Unity Catalog volumes as part of your bundle configuration.
Bundles:
* Add DABs support for Unity Catalog volumes ([#1762](#1762)).
* Support lookup by name of notification destinations ([#1922](#1922)).
* Extend "notebook not found" error to warn about missing extension ([#1920](#1920)).
* Skip sync warning if no sync paths are defined ([#1926](#1926)).
* Add validation for single node clusters ([#1909](#1909)).
* Fix segfault in bundle summary command ([#1937](#1937)).
* Add the `bundle_uuid` helper function for templates ([#1947](#1947)).
* Add default value for `volume_type` for DABs ([#1952](#1952)).
* Properly read Git metadata when running inside workspace ([#1945](#1945)).
* Upgrade TF provider to 1.59.0 ([#1960](#1960)).
Internal:
* Breakout variable lookup into separate files and tests ([#1921](#1921)).
* Add golangci-lint v1.62.2 ([#1953](#1953)).
Dependency updates:
* Bump golang.org/x/term from 0.25.0 to 0.26.0 ([#1907](#1907)).
* Bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1 ([#1930](#1930)).
* Bump github.com/stretchr/testify from 1.9.0 to 1.10.0 ([#1932](#1932)).
* Bump github.com/databricks/databricks-sdk-go from 0.51.0 to 0.52.0 ([#1931](#1931)).
@pieternpietern mentioned this pull request Dec 5, 2024
github-merge-queueBot pushed a commit that referenced this pull request Dec 5, 2024
**New features for Databricks Asset Bundles:**
This release adds support for managing Unity Catalog volumes as part of
your bundle configuration.
Bundles:
* Add DABs support for Unity Catalog volumes
([#1762](#1762)).
* Support lookup by name of notification destinations
([#1922](#1922)).
* Extend "notebook not found" error to warn about missing extension
([#1920](#1920)).
* Skip sync warning if no sync paths are defined
([#1926](#1926)).
* Add validation for single node clusters
([#1909](#1909)).
* Fix segfault in bundle summary command
([#1937](#1937)).
* Add the `bundle_uuid` helper function for templates
([#1947](#1947)).
* Add default value for `volume_type` for DABs
([#1952](#1952)).
* Properly read Git metadata when running inside workspace
([#1945](#1945)).
* Upgrade TF provider to 1.59.0
([#1960](#1960)).
Internal:
* Breakout variable lookup into separate files and tests
([#1921](#1921)).
* Add golangci-lint v1.62.2
([#1953](#1953)).
Dependency updates:
* Bump golang.org/x/term from 0.25.0 to 0.26.0
([#1907](#1907)).
* Bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1
([#1930](#1930)).
* Bump github.com/stretchr/testify from 1.9.0 to 1.10.0
([#1932](#1932)).
* Bump github.com/databricks/databricks-sdk-go from 0.51.0 to 0.52.0
([#1931](#1931)).
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
While looking into adding variable lookups for notification destinations
([API][API]), I found the codegen approach for different classes of
variable lookups a bit complex. The template had a custom field override
(for service principals), the package had an override for the cluster
lookup, and it didn't produce tests.
The notification destinations API uses a default page size of 20 for
listing. I want to use a larger page size to limit the number of API
calls, so that would imply another customization on the template or a
manual override.
This code being rather mechanical, I used copilot to produce all
instances of the resolvers and their tests (after writing one of them
manually).
[api]: https://docs.databricks.com/api/workspace/notificationdestinations
## Tests
* Unit tests pass
* Manual confirmation that lookups of warehouses still work
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Add support for notification destinations in variable lookups.
More information:
https://docs.databricks.com/en/admin/workspace-settings/notification-destinations.html
Depends on #1921.
## Tests
* New unit test
* Manually confirmed that the lookup works
denik pushed a commit that referenced this pull request May 20, 2026
**New features for Databricks Asset Bundles:**
This release adds support for managing Unity Catalog volumes as part of
your bundle configuration.
Bundles:
* Add DABs support for Unity Catalog volumes
([#1762](#1762)).
* Support lookup by name of notification destinations
([#1922](#1922)).
* Extend "notebook not found" error to warn about missing extension
([#1920](#1920)).
* Skip sync warning if no sync paths are defined
([#1926](#1926)).
* Add validation for single node clusters
([#1909](#1909)).
* Fix segfault in bundle summary command
([#1937](#1937)).
* Add the `bundle_uuid` helper function for templates
([#1947](#1947)).
* Add default value for `volume_type` for DABs
([#1952](#1952)).
* Properly read Git metadata when running inside workspace
([#1945](#1945)).
* Upgrade TF provider to 1.59.0
([#1960](#1960)).
Internal:
* Breakout variable lookup into separate files and tests
([#1921](#1921)).
* Add golangci-lint v1.62.2
([#1953](#1953)).
Dependency updates:
* Bump golang.org/x/term from 0.25.0 to 0.26.0
([#1907](#1907)).
* Bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1
([#1930](#1930)).
* Bump github.com/stretchr/testify from 1.9.0 to 1.10.0
([#1932](#1932)).
* Bump github.com/databricks/databricks-sdk-go from 0.51.0 to 0.52.0
([#1931](#1931)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pietern@eng-dev-ecosystem-bot@andrewnester