Skip to content

Switch default-python to use pyproject.toml & uv & dynamic_version in dev [hatchling backend] - #3042

Merged
denik merged 19 commits into
mainfrom
denik/pyproject-hatchling
Jul 2, 2025
Merged

Switch default-python to use pyproject.toml & uv & dynamic_version in dev [hatchling backend] #3042
denik merged 19 commits into
mainfrom
denik/pyproject-hatchling

Conversation

@denik

Copy link
Copy Markdown
Contributor

Changes

Why

Modern Python is based around pyproject.toml, not setup.py.

Using DAB-provided dynamic_version attribute allows keeping pyproject.toml simple and does not constraint what build backend users can choose.

Note, I've also considered using uv_build backend #3004 but it seems too early at the moment and hatchling is better tested with DABs.

Tests

Existing tests.

@denik
deniktemporarily deployed to test-trigger-is June 12, 2025 11:22 — with GitHub Actions Inactive
@denikdenik changed the title Switch default-python to use pyproject.toml & uv & dynamic_version in dev [hatchling] Switch default-python to use pyproject.toml & uv & dynamic_version in dev [hatchling backend] Jun 12, 2025
@denik
denikforce-pushed the denik/pyproject-hatchling branch from 84838b2 to 739f980CompareJune 12, 2025 11:26
@denik
deniktemporarily deployed to test-trigger-is June 12, 2025 11:26 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 12, 2025 11:27 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/pyproject-hatchling branch from a9fc9e8 to ecd0eaeCompareJune 12, 2025 11:29
@denik
deniktemporarily deployed to test-trigger-is June 12, 2025 11:29 — with GitHub Actions Inactive

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR modernizes the default‐python template by removing setup.py and in‐code versioning in favor of a pyproject.toml‐based hatchling build, switches the bundle to use uv build --wheel, and applies dynamic_version only on non‐serverless dev targets.

  • Drop __version__ and setup.py.tmpl, introduce pyproject.toml.tmpl with hatchling wheel config.
  • Update databricks.yml.tmpl to run uv build --wheel and conditionally enable dynamic_version for dev.
  • Refresh acceptance tests (out.*, check_output.py) and update NEXT_CHANGELOG.md.

Reviewed Changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
libs/template/.../init.pyRemove hardcoded __version__ assignment
libs/template/.../pyproject.toml.tmplAdd [project] metadata and hatchling wheel build config
libs/template/.../databricks.yml.tmplUse uv build --wheel and conditional dynamic_version in dev
acceptance/.../check_output.pyReplace literal upload check with regex-based UPLOADING_WHL
NEXT_CHANGELOG.mdDocument new default-python template changes
Comments suppressed due to low confidence (2)

NEXT_CHANGELOG.md:6

  • [nitpick] The backslash in dynamic\_version is only needed to escape Markdown italics. For clarity, wrap it in inline code: dynamic_version.
* Switch default-python template to use pyproject.toml + dynamic\_version in dev target. uv is now required. ([#3042](https://github.com/databricks/cli/pull/3042))

libs/template/templates/default-python/template/{{.project_name}}/databricks.yml.tmpl:8

  • The conditional artifacts: block under the dev target may be misaligned in indentation and could break YAML parsing. Align the artifacts: and python_artifact: keys with the workspace: block under dev.
{{ if $with_python }}


BUILDING = "Building python_artifact"
UPLOADING = "Uploading dist/"
UPLOADING_WHL = re.compile("Uploading .*whl")

CopilotAIJun 13, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In is_printable_line, using UPLOADING_WHL.match(line) only matches at the start and will miss indented or prefixed upload lines. Consider using search() in both the function and the regex (e.g., re.compile("Uploading .*\.whl")) or anchoring appropriately.

Suggested change
UPLOADING_WHL=re.compile("Uploading .*whl")
UPLOADING_WHL=re.compile(r"Uploading .*\.whl")

Copilot uses AI. Check for mistakes.
@denik
deniktemporarily deployed to test-trigger-is June 13, 2025 09:16 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/pyproject-hatchling branch from 63a504d to 039c825CompareJune 13, 2025 09:16
@denik
deniktemporarily deployed to test-trigger-is June 13, 2025 09:16 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/pyproject-hatchling branch from 039c825 to 206342eCompareJune 13, 2025 09:16
@denik
deniktemporarily deployed to test-trigger-is June 13, 2025 09:16 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 13, 2025 09:20 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/pyproject-hatchling branch from 77d07fa to 76ac9fbCompareJune 16, 2025 07:24
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 07:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 07:37 — with GitHub Actions Inactive

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 10:00 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 10:07 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 10:49 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 10:51 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is June 16, 2025 10:52 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/pyproject-hatchling branch from 0b9a8e3 to f26f5c6CompareJuly 1, 2025 16:04
@denik
deniktemporarily deployed to test-trigger-is July 1, 2025 16:04 — with GitHub Actions Inactive
denik added 19 commits July 2, 2025 11:35
…ot of dependencies, slows down "uv sync"
├── databricks-dlt v0.3.0 (extra: dev)
│ └── databricks-connect v15.4.9
│ ├── databricks-sdk v0.57.0
│ │ ├── google-auth v2.40.3
│ │ │ ├── cachetools v5.5.2
│ │ │ ├── pyasn1-modules v0.4.2
│ │ │ │ └── pyasn1 v0.6.1
│ │ │ └── rsa v4.9.1
│ │ │ └── pyasn1 v0.6.1
│ │ └── requests v2.32.4
│ │ ├── certifi v2025.6.15
│ │ ├── charset-normalizer v3.4.2
│ │ ├── idna v3.10
│ │ └── urllib3 v2.4.0
│ ├── googleapis-common-protos v1.70.0
│ │ └── protobuf v6.31.1
│ ├── grpcio v1.73.0
│ ├── grpcio-status v1.73.0
│ │ ├── googleapis-common-protos v1.70.0 (*)
│ │ ├── grpcio v1.73.0
│ │ └── protobuf v6.31.1
│ ├── numpy v1.26.4
│ ├── packaging v25.0
│ ├── pandas v2.3.0
│ │ ├── numpy v1.26.4
│ │ ├── python-dateutil v2.9.0.post0
│ │ │ └── six v1.17.0
│ │ ├── pytz v2025.2
│ │ └── tzdata v2025.2
│ ├── py4j v0.10.9.7
│ ├── pyarrow v20.0.0
│ ├── setuptools v80.9.0
│ └── six v1.17.0
@denik
denikforce-pushed the denik/pyproject-hatchling branch from f26f5c6 to a291214CompareJuly 2, 2025 09:35
@denik
deniktemporarily deployed to test-trigger-is July 2, 2025 09:35 — with GitHub Actions Inactive
@denik
denik merged commit a0580e4 into mainJul 2, 2025
@denik
denik deleted the denik/pyproject-hatchling branch July 2, 2025 10:55
deco-sdk-taggingBot added a commit that referenced this pull request Jul 2, 2025
## Release v0.258.0
### Notable Changes
* Switch default-python template to use pyproject.toml + dynamic\_version in dev target. uv is now required. ([#3042](#3042))
### Dependency updates
* Upgraded TF provider to 1.84.0 ([#3151](#3151))
### CLI
* Fixed an issue where running `databricks auth login` would remove the `cluster_id` field from profiles in `.databrickscfg`. The login process now preserves the `cluster_id` field. ([#2988](#2988))
### Bundles
* Added support for pipeline environment field ([#3153](#3153))
* "bundle summary" now prints diagnostic warnings to stderr ([#3123](#3123))
* "bundle open" will print the URL before opening the browser ([#3168](#3168))
denik added a commit that referenced this pull request May 20, 2026
denik added a commit that referenced this pull request May 20, 2026
## Changes
Add new bundle setting presets.artifacts_dynamic_version. If set to
true, all whl artifacts (in a given target) will get dynamic_version:
true setting.
## Why
We want to set this to true for all artifacts in all targets without
repeating every artifact under targets.
#3042
## Tests
New acceptance test.
---------
Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com>
denik added a commit that referenced this pull request May 20, 2026
… dev [hatchling backend] (#3042)
## Changes
- Update default-python template to use pyproject.toml and uv (with
hatchling).
- Get rid of dynamic version in setup.py and using dynamic_version
attribute on artifacts instead (only in dev target and only on
non-serverless clusters since serverless has fixed the issue with
versions not being updated in the platform). See
#2427#2520
## Why
Modern Python is based around pyproject.toml, not setup.py. Using DAB-provided dynamic_version attribute allows keeping
pyproject.toml simple and does not constraint what build backend users
can choose.
Note, I've also considered using uv_build backend
#3004 but it seems too early at
the moment and hatchling is better tested with DABs.
## Tests
Existing tests.
denik pushed a commit that referenced this pull request May 20, 2026
## Release v0.258.0
### Notable Changes
* Switch default-python template to use pyproject.toml + dynamic\_version in dev target. uv is now required. ([#3042](#3042))
### Dependency updates
* Upgraded TF provider to 1.84.0 ([#3151](#3151))
### CLI
* Fixed an issue where running `databricks auth login` would remove the `cluster_id` field from profiles in `.databrickscfg`. The login process now preserves the `cluster_id` field. ([#2988](#2988))
### Bundles
* Added support for pipeline environment field ([#3153](#3153))
* "bundle summary" now prints diagnostic warnings to stderr ([#3123](#3123))
* "bundle open" will print the URL before opening the browser ([#3168](#3168))
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@denik@kanterov@eng-dev-ecosystem-bot@pietern@andrewnester