Skip to content

Auto detect Python wheel packages and infer build command - #603

Merged
andrewnester merged 14 commits into
mainfrom
auto-detect
Jul 26, 2023
Merged

Auto detect Python wheel packages and infer build command#603
andrewnester merged 14 commits into
mainfrom
auto-detect

Conversation

@andrewnester

@andrewnesterandrewnester commented Jul 24, 2023

Copy link
Copy Markdown
Contributor

Changes

Auto detect Python wheel packages and infer build command

Tests

bundle.yml

bundle:
name: wheel-task
workspace:
host: ****
resources:
jobs:
test_job:
name: "[${bundle.environment}] My Wheel Job"
tasks:
- task_key: TestTask
existing_cluster_id: "0123-aaaaaa-bbbbbbbbb"
python_wheel_task:
package_name: "my_test_code"
entry_point: "run"
libraries:
- whl: ./dist/my_test_code-*.whl

andrew.nester@HFW9Y94129 wheel % cli bundle deploy
artifacts.whl.AutoDetect: Detecting Python wheel project...
artifacts.whl.AutoDetect: Found Python wheel project at /Users/andrew.nester/dabs/wheel
artifacts.whl.Build(my_test_code): Building...
artifacts.whl.Build(my_test_code): Build succeeded
Starting upload of bundle files
Uploaded bundle files at /Users/andrew.nester@databricks.com/.bundle/wheel-task/default/files!
artifacts.Upload(my_test_code-0.0.0a0-py3-none-any.whl): Uploading...
artifacts.Upload(my_test_code-0.0.0a0-py3-none-any.whl): Upload succeeded
Starting resource deployment
Resource deployment completed!

pietern
pietern reviewed Jul 25, 2023
View reviewed changes

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Do you want to hold until the other PR is merged?

Comment threadbundle/artifacts/autodetect.go
Comment threadbundle/artifacts/whl/autodetect.go Outdated
@andrewnester

andrewnester commented Jul 25, 2023

Copy link
Copy Markdown
ContributorAuthor

@pietern yeah, let's wait for the other PR to be merged first

@andrewnester
andrewnester changed the base branch from artifact-block to mainJuly 25, 2023 11:35
pietern
pietern approved these changes Jul 25, 2023
View reviewed changes

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a few unit tests before merging.

Comment threadbundle/artifacts/infer.go

if len(artifact.Files) > 0 || artifact.BuildCommand != "" {
return nil
}

@pieternpieternJul 25, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment here -- I had to scroll back and forth a couple times before figuring out that we never build an artifact if the files are explicitly listed.

@andrewnesterandrewnesterJul 26, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, we should not build only if there is no BuildCommand

Comment threadbundle/artifacts/whl/autodetect.go
@andrewnester
andrewnester enabled auto-merge (squash) July 26, 2023 10:03
@andrewnester
andrewnester merged commit cfff140 into mainJul 26, 2023
@mgyuchtmgyucht mentioned this pull request Jul 27, 2023
Merged
github-merge-queueBot pushed a commit that referenced this pull request Jul 27, 2023
@mgyucht
Release v0.202.0 (#619)
3697dfc
Breaking Change:
* Require include glob patterns to be explicitly defined
([#602](#602)).
Bundles:
* Add support for more SDK config options
([#587](#587)).
* Add template renderer for Databricks templates
([#589](#589)).
* Fix formatting in renderer.go
([#593](#593)).
* Fixed python wheel test
([#608](#608)).
* Auto detect Python wheel packages and infer build command
([#603](#603)).
* Added support for artifacts building for bundles
([#583](#583)).
* Add support for cloning repositories
([#544](#544)).
* Add regexp compile helper function for templates
([#601](#601)).
* Add unit test that raw strings are printed as is
([#599](#599)).
Internal:
* Fix tests under ./cmd/configure if DATABRICKS_TOKEN is set
([#605](#605)).
* Remove dependency on global state in generated commands
([#595](#595)).
* Remove dependency on global state for the root command
([#606](#606)).
* Add merge_group trigger for build
([#612](#612)).
* Added support for build command chaining and error on missing wheel
([#607](#607)).
* Add TestAcc prefix to filer test and fix any failing tests
([#611](#611)).
* Add url parse helper function for templates
([#600](#600)).
* Remove dependency on global state for remaining commands
([#613](#613)).
* Update CHANGELOG template
([#588](#588)).
@PaulMaksud

PaulMaksud commented Aug 16, 2023•
edited
Loading

Copy link
Copy Markdown

Hello on version Databricks CLI v0.203.1 on windows intel 64 i face this error :
$ databricks bundle deploy -e dev artifacts.whl.AutoDetect: Detecting Python wheel project... artifacts.whl.AutoDetect: Found Python wheel project at C:\Users\Sesa653627\Desktop\ml integ\pilot-databricks artifacts.whl.Build(artifact1692195769): Building... Error: artifacts.whl.Build(artifact1692195769): Failed exit status 9009, output: Python est introuvable. Ex�cutez sans argument pour proc�der � l
even though i have no python wheel task declared in my bundle.yml, however i do have a setup.py file at root level and a pyproject.toml file

@pietern

pietern commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

@PaulMaksud Thanks for reporting.

Could you create a new issue for this, and include the (redacted) contents of your setup.py file?

@pietern
pietern deleted the auto-detect branch August 16, 2023 14:32
@jusshenjusshen mentioned this pull request Aug 23, 2023
Closed
denik pushed a commit that referenced this pull request May 20, 2026
@andrewnester
Auto detect Python wheel packages and infer build command (#603)
2ffeca8
denik pushed a commit that referenced this pull request May 20, 2026
@mgyucht
Release v0.202.0 (#619)
29c2a5b
Breaking Change:
* Require include glob patterns to be explicitly defined
([#602](#602)).
Bundles:
* Add support for more SDK config options
([#587](#587)).
* Add template renderer for Databricks templates
([#589](#589)).
* Fix formatting in renderer.go
([#593](#593)).
* Fixed python wheel test
([#608](#608)).
* Auto detect Python wheel packages and infer build command
([#603](#603)).
* Added support for artifacts building for bundles
([#583](#583)).
* Add support for cloning repositories
([#544](#544)).
* Add regexp compile helper function for templates
([#601](#601)).
* Add unit test that raw strings are printed as is
([#599](#599)).
Internal:
* Fix tests under ./cmd/configure if DATABRICKS_TOKEN is set
([#605](#605)).
* Remove dependency on global state in generated commands
([#595](#595)).
* Remove dependency on global state for the root command
([#606](#606)).
* Add merge_group trigger for build
([#612](#612)).
* Added support for build command chaining and error on missing wheel
([#607](#607)).
* Add TestAcc prefix to filer test and fix any failing tests
([#611](#611)).
* Add url parse helper function for templates
([#600](#600)).
* Remove dependency on global state for remaining commands
([#613](#613)).
* Update CHANGELOG template
([#588](#588)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pieternpieternpietern approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewnester@PaulMaksud@pietern