Uh oh!
There was an error while loading. Please reload this page.
Fix bundle generate job to preserve nested notebook directory structure - #4596
Merged
andrewnester merged 8 commits intoMay 8, 2026
Merged
Conversation
varundeepsaini
requested review from
andrewnester, anton-107, denik, pietern and shreyas-goenka
as code ownersFebruary 25, 2026 15:43
andrewnester
approved these changes
Feb 27, 2026
Collaborator
Commit: baac699
17 interesting tests: 10 SKIP, 7 KNOWN
Top 20 slowest tests (at least 2 minutes):
|
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
February 27, 2026 17:08
dc572e1 to
6a23c44Comparevarundeepsaini
commented
Mar 3, 2026
ContributorAuthor
@andrewnester could you run the checks, there were some issues on windows, have fixed those |
andrewnester
approved these changes
Mar 3, 2026
andrewnester
approved these changes
Mar 3, 2026
andrewnester
left a comment
Contributor
There was a problem hiding this comment.
Could you please add a NEXT_CHANGELOG entry for this change, following the format there? Thank you!
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
March 3, 2026 09:46
6a23c44 to
545af5cComparevarundeepsaini
commented
Mar 3, 2026
ContributorAuthor
@andrewnester done. |
andrewnester
self-requested a review
March 4, 2026 16:11
andrewnester
commented
Mar 4, 2026
Contributor
@varundeepsaini, this change could break for users who continuously run the "bundle generates job" command to sync the files back. At the very least, can you expand this PR to clean up the files at the old location? |
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
2 times, most recently
from
March 4, 2026 17:16
2e1b40d to
7f9bca0Comparevarundeepsaini
commented
Mar 4, 2026
ContributorAuthor
@andrewnester could you run the ci ? |
andrewnester
commented
Mar 11, 2026
Contributor
@varundeepsaini sorry for the delay, could you please also resolve merge conflicts? |
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
March 11, 2026 10:38
7f9bca0 to
8a557b2Comparevarundeepsaini
commented
Mar 11, 2026
ContributorAuthor
@andrewnester no worries, thanks a lot for the reviews |
Uh oh!
There was an error while loading. Please reload this page.
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
March 11, 2026 10:45
8a557b2 to
4876824Compareandrewnester
approved these changes
Mar 16, 2026
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
March 17, 2026 14:52
d7db5b1 to
454eb63Comparevarundeepsaini
commented
Mar 17, 2026
ContributorAuthor
@andrewnester I have fixed the windows test failure, |
varundeepsaini
commented
Mar 27, 2026
ContributorAuthor
@andrewnester i fixed the test failures, could you re run the ci |
auto-merge was automatically disabled
March 27, 2026 14:42
Head branch was pushed to by a user without write access
varundeepsainiforce-pushed
the
fix/bundle-generate-job
branch
from
March 27, 2026 14:42
1f9e016 to
baac699Comparesimonfaltum
removed their request for review
April 8, 2026 12:27
varundeepsaini
commented
Apr 8, 2026
ContributorAuthor
@andrewnester |
varundeepsaini
commented
Apr 9, 2026
ContributorAuthor
@andrewnester during env setup |
varundeepsaini
commented
Apr 15, 2026
ContributorAuthor
@andrewnester could you run the ci |
varundeepsaini
commented
Apr 21, 2026
ContributorAuthor
@andrewnester bumping this ^^ |
andrewnester
commented
Apr 22, 2026
Contributor
@varundeepsaini unfortunately right now CI actions do not work on PRs created from forks, we're working on it |
varundeepsaini
commented
Apr 22, 2026
ContributorAuthor
@andrewnester no worries, thanks for the heads up |
Contributor
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
varundeepsaini
commented
May 8, 2026
ContributorAuthor
@andrewnester could you approve the ci run |
Uh oh!
There was an error while loading. Please reload this page.
2 tasks
pietern added a commit
that referenced
this pull request
May 11, 2026
## Summary - PR #4596 (just merged on main) committed `out.test.toml` for `acceptance/bundle/generate/job_nested_notebooks` in the pre-#5146 multi-line `[EnvMatrix]` form. - Regenerated via `go test ./acceptance -run '^TestAccept$' -only-out-test-toml`. ## Test plan - [x] `go test ./acceptance -run '^TestAccept$' -only-out-test-toml` produces only this one-file diff - [x] `git diff` after regeneration matches the diff CI was reporting This pull request and its description were written by Isaac.
deco-sdk-taggingBot
added a commit
that referenced
this pull request
May 13, 2026
## Release v0.299.2 ### Notable Changes * Breaking change: `vector_search_endpoints` renamed `min_qps` to `target_qps` in DABs configuration and the `vector-search-endpoints` commands, following the SDK rename in v0.131.0. Update any `databricks.yml` using `min_qps:` to `target_qps:` and any CLI invocations using `--min-qps` to `--target-qps`. ### CLI * `auth login` no longer falls back to plaintext when the OS keyring is reachable but locked. The unlock prompt shown by the probe now runs in parallel with the OAuth flow, and the token is stored in the keyring once the user has typed their password. * `databricks auth describe` now reports where U2M (`databricks-cli`) tokens are stored: `plaintext` (`~/.databricks/token-cache.json`) or `secure` (OS keyring), and the source of the choice (env var, config setting, or default). * Marked the default profile in the interactive pickers shown by `databricks auth switch`, `databricks auth logout`, `databricks auth token`, and `databricks auth login`, and moved it to the top of the list. `databricks auth login` and `databricks auth logout` now offer the same selectors as `databricks auth token` and `databricks auth switch` respectively. * The interactive auth profile pickers now start in search mode so typing immediately filters the list, and the action entries (`+ Create a new profile`, `→ Enter a host URL manually`) are visually distinct from real profiles and stay visible regardless of the search query. * Shortened the host prompt label shown after `→ Enter a host URL manually` in `databricks auth login` so the prompt no longer leaves stale lines on screen when typing or pasting a host URL. ### Bundles * Stop applying `presets.name_prefix` (and the dev-mode `[dev <user>]` rename) to `vector_search_endpoints` ([#5209](#5209)). * Fix `bundle generate` job to preserve nested notebook directory structure ([#4596](#4596)) * Propagate authentication environment (including `DATABRICKS_CONFIG_PROFILE`) to the `experimental.python` subprocess so bundle validate/deploy no longer fails with a multi-profile host ambiguity error when several profiles in `~/.databrickscfg` share the same host. * Fixed `--force-pull` on `bundle summary` and `bundle open` so the flag bypasses the local state cache and reads state from the workspace. ### Dependency updates * Bump Go toolchain to 1.25.10 ([#5213](#5213)). * Bump `github.com/databricks/databricks-sdk-go` from v0.128.0 to v0.132.0. * Bump Terraform provider to v1.115.0.
denik pushed a commit
that referenced
this pull request
May 20, 2026
…re (#4596) ## Changes Fixes#4503. Add `MarkTasksForDownload` that computes a common base path across all notebook tasks before downloading, replacing the per-task loop in `job.go`. This preserves nested directory structure instead of flattening all notebooks into `src/`. ## Tests - Unit tests for `commonDirPrefix` and `MarkTasksForDownload`. - Acceptance test `bundle/generate/job_nested_notebooks` reproducing the issue. - No regressions in existing `python_job`, `ipynb_job`, `git_job` tests. Test are written by Claude Code Co-authored-by: Andrew Nester <andrew.nester.dev@gmail.com>
denik pushed a commit
that referenced
this pull request
May 20, 2026
## Summary - PR #4596 (just merged on main) committed `out.test.toml` for `acceptance/bundle/generate/job_nested_notebooks` in the pre-#5146 multi-line `[EnvMatrix]` form. - Regenerated via `go test ./acceptance -run '^TestAccept$' -only-out-test-toml`. ## Test plan - [x] `go test ./acceptance -run '^TestAccept$' -only-out-test-toml` produces only this one-file diff - [x] `git diff` after regeneration matches the diff CI was reporting This pull request and its description were written by Isaac.
denik pushed a commit
that referenced
this pull request
May 20, 2026
## Release v0.299.2 ### Notable Changes * Breaking change: `vector_search_endpoints` renamed `min_qps` to `target_qps` in DABs configuration and the `vector-search-endpoints` commands, following the SDK rename in v0.131.0. Update any `databricks.yml` using `min_qps:` to `target_qps:` and any CLI invocations using `--min-qps` to `--target-qps`. ### CLI * `auth login` no longer falls back to plaintext when the OS keyring is reachable but locked. The unlock prompt shown by the probe now runs in parallel with the OAuth flow, and the token is stored in the keyring once the user has typed their password. * `databricks auth describe` now reports where U2M (`databricks-cli`) tokens are stored: `plaintext` (`~/.databricks/token-cache.json`) or `secure` (OS keyring), and the source of the choice (env var, config setting, or default). * Marked the default profile in the interactive pickers shown by `databricks auth switch`, `databricks auth logout`, `databricks auth token`, and `databricks auth login`, and moved it to the top of the list. `databricks auth login` and `databricks auth logout` now offer the same selectors as `databricks auth token` and `databricks auth switch` respectively. * The interactive auth profile pickers now start in search mode so typing immediately filters the list, and the action entries (`+ Create a new profile`, `→ Enter a host URL manually`) are visually distinct from real profiles and stay visible regardless of the search query. * Shortened the host prompt label shown after `→ Enter a host URL manually` in `databricks auth login` so the prompt no longer leaves stale lines on screen when typing or pasting a host URL. ### Bundles * Stop applying `presets.name_prefix` (and the dev-mode `[dev <user>]` rename) to `vector_search_endpoints` ([#5209](#5209)). * Fix `bundle generate` job to preserve nested notebook directory structure ([#4596](#4596)) * Propagate authentication environment (including `DATABRICKS_CONFIG_PROFILE`) to the `experimental.python` subprocess so bundle validate/deploy no longer fails with a multi-profile host ambiguity error when several profiles in `~/.databrickscfg` share the same host. * Fixed `--force-pull` on `bundle summary` and `bundle open` so the flag bypasses the local state cache and reads state from the workspace. ### Dependency updates * Bump Go toolchain to 1.25.10 ([#5213](#5213)). * Bump `github.com/databricks/databricks-sdk-go` from v0.128.0 to v0.132.0. * Bump Terraform provider to v1.115.0.
TanishqDatabricks pushed a commit
to TanishqDatabricks/cli
that referenced
this pull request
May 22, 2026
…re (databricks#4596) ## Changes Fixesdatabricks#4503. Add `MarkTasksForDownload` that computes a common base path across all notebook tasks before downloading, replacing the per-task loop in `job.go`. This preserves nested directory structure instead of flattening all notebooks into `src/`. ## Tests - Unit tests for `commonDirPrefix` and `MarkTasksForDownload`. - Acceptance test `bundle/generate/job_nested_notebooks` reproducing the issue. - No regressions in existing `python_job`, `ipynb_job`, `git_job` tests. Test are written by Claude Code Co-authored-by: Andrew Nester <andrew.nester.dev@gmail.com>
TanishqDatabricks pushed a commit
to TanishqDatabricks/cli
that referenced
this pull request
May 22, 2026
…cks#5228) ## Summary - PR databricks#4596 (just merged on main) committed `out.test.toml` for `acceptance/bundle/generate/job_nested_notebooks` in the pre-databricks#5146 multi-line `[EnvMatrix]` form. - Regenerated via `go test ./acceptance -run '^TestAccept$' -only-out-test-toml`. ## Test plan - [x] `go test ./acceptance -run '^TestAccept$' -only-out-test-toml` produces only this one-file diff - [x] `git diff` after regeneration matches the diff CI was reporting This pull request and its description were written by Isaac.
TanishqDatabricks pushed a commit
to TanishqDatabricks/cli
that referenced
this pull request
May 22, 2026
## Release v0.299.2 ### Notable Changes * Breaking change: `vector_search_endpoints` renamed `min_qps` to `target_qps` in DABs configuration and the `vector-search-endpoints` commands, following the SDK rename in v0.131.0. Update any `databricks.yml` using `min_qps:` to `target_qps:` and any CLI invocations using `--min-qps` to `--target-qps`. ### CLI * `auth login` no longer falls back to plaintext when the OS keyring is reachable but locked. The unlock prompt shown by the probe now runs in parallel with the OAuth flow, and the token is stored in the keyring once the user has typed their password. * `databricks auth describe` now reports where U2M (`databricks-cli`) tokens are stored: `plaintext` (`~/.databricks/token-cache.json`) or `secure` (OS keyring), and the source of the choice (env var, config setting, or default). * Marked the default profile in the interactive pickers shown by `databricks auth switch`, `databricks auth logout`, `databricks auth token`, and `databricks auth login`, and moved it to the top of the list. `databricks auth login` and `databricks auth logout` now offer the same selectors as `databricks auth token` and `databricks auth switch` respectively. * The interactive auth profile pickers now start in search mode so typing immediately filters the list, and the action entries (`+ Create a new profile`, `→ Enter a host URL manually`) are visually distinct from real profiles and stay visible regardless of the search query. * Shortened the host prompt label shown after `→ Enter a host URL manually` in `databricks auth login` so the prompt no longer leaves stale lines on screen when typing or pasting a host URL. ### Bundles * Stop applying `presets.name_prefix` (and the dev-mode `[dev <user>]` rename) to `vector_search_endpoints` ([databricks#5209](databricks#5209)). * Fix `bundle generate` job to preserve nested notebook directory structure ([databricks#4596](databricks#4596)) * Propagate authentication environment (including `DATABRICKS_CONFIG_PROFILE`) to the `experimental.python` subprocess so bundle validate/deploy no longer fails with a multi-profile host ambiguity error when several profiles in `~/.databrickscfg` share the same host. * Fixed `--force-pull` on `bundle summary` and `bundle open` so the flag bypasses the local state cache and reads state from the workspace. ### Dependency updates * Bump Go toolchain to 1.25.10 ([databricks#5213](databricks#5213)). * Bump `github.com/databricks/databricks-sdk-go` from v0.128.0 to v0.132.0. * Bump Terraform provider to v1.115.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Fixes#4503.
Add
MarkTasksForDownloadthat computes a common base path across all notebook tasks before downloading, replacing the per-task loop injob.go. This preserves nested directory structure instead of flattening all notebooks intosrc/.Tests
commonDirPrefixandMarkTasksForDownload.bundle/generate/job_nested_notebooksreproducing the issue.python_job,ipynb_job,git_jobtests.Test are written by Claude Code