Raise an error when there are multiple local libraries with the same basename used by andrewnester · Pull Request #2382 · databricks/cli

andrewnester · 2025-02-26T11:38:07Z

Changes

It could happen that there are multiple artifacts defined in the bundle which build and therefore deploy wheel packages with the same name. This leads to conflict between these packages, they will overwrite each other and therefore they should have different names instead

Fixes#1674

Previous attempt (#2297 + #2341) led to the breakage, this PR fixes both issues.

Tests

Added acceptance test

…asename are used

andrewnester · 2025-02-26T11:42:38Z

+ var err error
+ for _, pattern := range patterns {
+ v, err = dyn.MapByPattern(v, pattern, func(p dyn.Path, lv dyn.Value) (dyn.Value, error) {
+ libPath, ok := lv.AsString()


Changed from MustString to AsString to avoid potential panic which led to the breakage before

andrewnester · 2025-02-26T11:42:58Z

+ v, err = dyn.MapByPattern(v, pattern, func(p dyn.Path, lv dyn.Value) (dyn.Value, error) {
+ libPath, ok := lv.AsString()
+ if !ok {
+ return lv, nil


return correct map here (lv instead of v) which lead to the issue before

## Changes Added PyPi and Maven libraries tests Needed for this PR since we don't currently have any coverage for PyPi or Maven libraries #2382

shreyas-goenka · 2025-02-27T15:14:30Z

+
+type checkForSameNameLibraries struct{}
+
+var patterns = []dyn.Pattern{


We can inline this since it's only used in one location?

shreyas-goenka · 2025-02-27T15:32:13Z

+ return libraryValue, nil
+ }
+
+ lib := filepath.Base(libPath)


nit:
Suggested change
lib:=filepath.Base(libPath)
name:=filepath.Base(libPath)

pietern · 2025-02-28T09:34:08Z

+var patterns = []dyn.Pattern{
+	taskLibrariesPattern.Append(dyn.AnyIndex(), dyn.AnyKey()),
+	forEachTaskLibrariesPattern.Append(dyn.AnyIndex(), dyn.AnyKey()),
+	envDepsPattern.Append(dyn.AnyIndex()),


Q: this matches on any key, but we know it only works for "whl" and "jar" (and maybe more).
Could the pattern be more specific to reflect this?

Good point, followed up here: #2401

) ## Changes Added PyPi and Maven libraries tests Needed for this PR since we don't currently have any coverage for PyPi or Maven libraries databricks#2382

…basename used (databricks#2382) ## Changes It could happen that there are multiple artifacts defined in the bundle which build and therefore deploy wheel packages with the same name. This leads to conflict between these packages, they will overwrite each other and therefore they should have different names instead Fixesdatabricks#1674 Previous attempt (databricks#2297 + databricks#2341) led to the breakage, this PR fixes both issues. ## Tests Added acceptance test

CLI: * Upgrade Go SDK to 0.59.0 ([#2425](#2425)). Bundles: * Added a warning when `config` section is used in apps ([#2416](#2416)). * Switch to use GET workspaces-files/{name} instead of workspace/export for state files to avoid 10MB limit ([#2423](#2423)). * Use schema field for pipeline in builtin template ([#2347](#2347)). * Add warning when variable interpolation is used for auth fields ([#2399](#2399)). * Add warning when include is used in config files other than databricks.yml ([#2389](#2389)). * Add support for schemas in deployment bind/unbind commands ([#2406](#2406)). * Do not modify/create .gitignore in bundle root ([#2429](#2429)). * Raise an error when there are multiple local libraries with the same basename used ([#2382](#2382)). * Upgrade TF provider to 1.68.0 ([#2426](#2426)). API Changes: * Changed `databricks experiments log-inputs` command with new required argument order. * Added `databricks genie get-space` command. * Added `databricks providers list-provider-share-assets` command. * Changed `databricks shares update-permissions` command return type to become non-empty. OpenAPI commit e5c870006a536121442cfd2441bdc8a5fb76ae1e (2025-03-03)

## Changes Added PyPi and Maven libraries tests Needed for this PR since we don't currently have any coverage for PyPi or Maven libraries #2382

…basename used (#2382) ## Changes It could happen that there are multiple artifacts defined in the bundle which build and therefore deploy wheel packages with the same name. This leads to conflict between these packages, they will overwrite each other and therefore they should have different names instead Fixes#1674 Previous attempt (#2297 + #2341) led to the breakage, this PR fixes both issues. ## Tests Added acceptance test

CLI: * Upgrade Go SDK to 0.59.0 ([#2425](#2425)). Bundles: * Added a warning when `config` section is used in apps ([#2416](#2416)). * Switch to use GET workspaces-files/{name} instead of workspace/export for state files to avoid 10MB limit ([#2423](#2423)). * Use schema field for pipeline in builtin template ([#2347](#2347)). * Add warning when variable interpolation is used for auth fields ([#2399](#2399)). * Add warning when include is used in config files other than databricks.yml ([#2389](#2389)). * Add support for schemas in deployment bind/unbind commands ([#2406](#2406)). * Do not modify/create .gitignore in bundle root ([#2429](#2429)). * Raise an error when there are multiple local libraries with the same basename used ([#2382](#2382)). * Upgrade TF provider to 1.68.0 ([#2426](#2426)). API Changes: * Changed `databricks experiments log-inputs` command with new required argument order. * Added `databricks genie get-space` command. * Added `databricks providers list-provider-share-assets` command. * Changed `databricks shares update-permissions` command return type to become non-empty. OpenAPI commit e5c870006a536121442cfd2441bdc8a5fb76ae1e (2025-03-03)

andrewnester added 5 commits February 26, 2025 12:29

Raise an error when there are multiple locallibraries with the same b…
f10b6e4
…asename are used

added missing comment
0391c56

fixes
aaf74bf

added acc test
5910818

fixes
a379fb3

andrewnester added the do-not-merge label Feb 26, 2025

andrewnester requested review from denik, pietern and shreyas-goenka as code owners February 26, 2025 11:38

andrewnester temporarily deployed to test-trigger-is February 26, 2025 11:38 — with GitHub Actions Inactive

andrewnester mentioned this pull request Feb 26, 2025
acc: Added tests for deploying PyPi and Maven libraries #2359
Merged

andrewnester commented Feb 26, 2025
View reviewed changes

andrewnester added a commit that referenced this pull request Feb 26, 2025
acc: Added tests for deploying PyPi and Maven libraries (#2359)
fa79d04
## Changes Added PyPi and Maven libraries tests Needed for this PR since we don't currently have any coverage for PyPi or Maven libraries #2382

andrewnester removed the do-not-merge label Feb 26, 2025

Merge branch 'main' into fix/same-name-wheels
f7359d0

andrewnester temporarily deployed to test-trigger-is February 26, 2025 14:56 — with GitHub Actions Inactive

denik reviewed Feb 27, 2025
View reviewed changes

added acc test + cleanup
d933227

andrewnester requested a review from denik February 27, 2025 13:21

Merge branch 'main' into fix/same-name-wheels
4e2557c

andrewnester temporarily deployed to test-trigger-is February 27, 2025 13:22 — with GitHub Actions Inactive

denik reviewed Feb 27, 2025
View reviewed changes

Comment threadacceptance/bundle/artifacts/unique_name_libraries/test.toml Outdated

added full paths
3360d78

andrewnester temporarily deployed to test-trigger-is February 27, 2025 14:26 — with GitHub Actions Inactive

andrewnester requested a review from denik February 27, 2025 14:26

andrewnester enabled auto-merge February 27, 2025 14:27

added repls for Windows slashes
3224b00

andrewnester temporarily deployed to test-trigger-is February 27, 2025 15:13 — with GitHub Actions Inactive

fix typo
6cd1670

andrewnester temporarily deployed to test-trigger-is February 27, 2025 15:31 — with GitHub Actions Inactive

shreyas-goenka approved these changes Feb 27, 2025
View reviewed changes

fix
085b74b

andrewnester temporarily deployed to test-trigger-is February 27, 2025 15:45 — with GitHub Actions Inactive

repl in a correct file
b73c78a

andrewnester temporarily deployed to test-trigger-is February 27, 2025 16:10 — with GitHub Actions Inactive

andrewnester added this pull request to the merge queue Feb 27, 2025

Merged via the queue into main with commit 6a07e05Feb 27, 2025

andrewnester deleted the fix/same-name-wheels branch February 27, 2025 16:41

pietern reviewed Feb 28, 2025
View reviewed changes

andrewnester mentioned this pull request Mar 5, 2025
[Release] Release v0.243.0 #2435
Merged

denik pushed a commit that referenced this pull request May 20, 2026
acc: Added tests for deploying PyPi and Maven libraries (#2359)
36c75f4
## Changes Added PyPi and Maven libraries tests Needed for this PR since we don't currently have any coverage for PyPi or Maven libraries #2382

Navigation Menu

Uh oh!

Raise an error when there are multiple local libraries with the same basename used - #2382

Raise an error when there are multiple local libraries with the same basename used #2382
andrewnester merged 13 commits into
mainfrom
fix/same-name-wheels

andrewnester commented Feb 26, 2025•
edited
Loading

Uh oh!

andrewnesterFeb 26, 2025

Uh oh!

andrewnesterFeb 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shreyas-goenkaFeb 27, 2025

Uh oh!

shreyas-goenkaFeb 27, 2025

Uh oh!

pieternFeb 28, 2025

Uh oh!

andrewnesterFeb 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		type checkForSameNameLibraries struct{}

		var patterns = []dyn.Pattern{

Uh oh!

Conversation

andrewnester commented Feb 26, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

andrewnesterFeb 26, 2025

Choose a reason for hiding this comment

Uh oh!

andrewnesterFeb 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shreyas-goenkaFeb 27, 2025

Choose a reason for hiding this comment

Uh oh!

shreyas-goenkaFeb 27, 2025

Choose a reason for hiding this comment

Uh oh!

pieternFeb 28, 2025

Choose a reason for hiding this comment

Uh oh!

andrewnesterFeb 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andrewnester commented Feb 26, 2025•
edited
Loading